Compare commits

...

32 Commits

Author SHA1 Message Date
04a90469fe Remove .gitlab-ci.yml
All checks were successful
continuous-integration/drone/push Build is passing
2021-04-19 09:01:06 +02:00
117a052aa5 Merge pull request #259 from vbrandl/dependabot/cargo/futures-0.3.14
Bump futures from 0.3.13 to 0.3.14
2021-04-19 08:59:59 +02:00
b2bc7fc217 Bump futures from 0.3.13 to 0.3.14
Bumps [futures](https://github.com/rust-lang/futures-rs) from 0.3.13 to 0.3.14.
- [Release notes](https://github.com/rust-lang/futures-rs/releases)
- [Changelog](https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/futures-rs/compare/0.3.13...0.3.14)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2021-04-13 17:58:36 +00:00
cd2624ae11 Merge pull request #256 from vbrandl/dependabot/cargo/tracing-bunyan-formatter-0.2.0
Bump tracing-bunyan-formatter from 0.1.7 to 0.2.0
2021-04-13 19:56:58 +02:00
fa9601bb9a Merge pull request #258 from vbrandl/dependabot/cargo/vergen-5.1.1
Bump vergen from 5.1.0 to 5.1.1
2021-04-13 19:55:14 +02:00
ae9d31b82c Bump vergen from 5.1.0 to 5.1.1
Bumps [vergen](https://github.com/rustyhorde/vergen) from 5.1.0 to 5.1.1.
- [Release notes](https://github.com/rustyhorde/vergen/releases)
- [Commits](https://github.com/rustyhorde/vergen/compare/5.1.0...5.1.1)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2021-04-08 04:18:02 +00:00
2afe5fc172 Bump tracing-bunyan-formatter from 0.1.7 to 0.2.0
Bumps [tracing-bunyan-formatter](https://github.com/LukeMathWalker/tracing-bunyan-formatter) from 0.1.7 to 0.2.0.
- [Release notes](https://github.com/LukeMathWalker/tracing-bunyan-formatter/releases)
- [Commits](https://github.com/LukeMathWalker/tracing-bunyan-formatter/commits/v0.2.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2021-03-30 04:19:49 +00:00
ea7f074661 Merge pull request #244 from vbrandl/dependabot/cargo/ructe-0.13.2
Bump ructe from 0.13.0 to 0.13.2
2021-03-29 14:45:53 +02:00
d5e3cad299 Bump ructe from 0.13.0 to 0.13.2
Bumps [ructe](https://github.com/kaj/ructe) from 0.13.0 to 0.13.2.
- [Release notes](https://github.com/kaj/ructe/releases)
- [Changelog](https://github.com/kaj/ructe/blob/master/CHANGELOG.md)
- [Commits](https://github.com/kaj/ructe/compare/v0.13.0...v0.13.2)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2021-03-29 10:56:11 +00:00
6570daef02 Remove macos from test matrix
Some checks failed
continuous-integration/drone/push Build is failing
2021-03-29 12:55:49 +02:00
6ac389e3ee Merge pull request #239 from vbrandl/dependabot/cargo/serde_json-1.0.64
Bump serde_json from 1.0.63 to 1.0.64
2021-03-29 12:55:14 +02:00
36196975ac Merge pull request #251 from vbrandl/dependabot/cargo/vergen-5.1.0
Bump vergen from 4.1.0 to 5.1.0
2021-03-29 12:54:28 +02:00
1dd83829b2 Bump serde_json from 1.0.63 to 1.0.64
Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.63 to 1.0.64.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.63...v1.0.64)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2021-03-29 10:48:28 +00:00
1b0817bc25 Bump vergen from 4.1.0 to 5.1.0
All checks were successful
continuous-integration/drone/push Build is passing
Bumps [vergen](https://github.com/rustyhorde/vergen) from 4.1.0 to 5.1.0.
- [Release notes](https://github.com/rustyhorde/vergen/releases)
- [Commits](https://github.com/rustyhorde/vergen/compare/4.1.0...5.1.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2021-03-29 12:45:49 +02:00
f57dc8e890 Fix vergen 2021-03-29 12:45:49 +02:00
bdda6292d2 Bump vergen from 4.1.0 to 5.1.0
Bumps [vergen](https://github.com/rustyhorde/vergen) from 4.1.0 to 5.1.0.
- [Release notes](https://github.com/rustyhorde/vergen/releases)
- [Commits](https://github.com/rustyhorde/vergen/compare/4.1.0...5.1.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2021-03-29 12:45:49 +02:00
a2826c9922 Remove windows from test matrix
Some checks failed
continuous-integration/drone/push Build is failing
2021-03-29 12:44:55 +02:00
3b2a368df0 Merge pull request #247 from vbrandl/dependabot/cargo/config-0.11.0
Bump config from 0.10.1 to 0.11.0
2021-03-29 12:23:55 +02:00
8b7444bd7c Merge pull request #253 from vbrandl/feature/refactor2
Feature/refactor2
2021-03-29 12:12:58 +02:00
9d719e7d85 Merge pull request #252 from vbrandl/master
Refactor for testability
2021-03-29 12:12:09 +02:00
584379e047 Merge pull request #250 from vbrandl/dependabot/cargo/serde-1.0.125
Bump serde from 1.0.123 to 1.0.125
2021-03-29 12:10:50 +02:00
951124e038 Bump serde from 1.0.123 to 1.0.125
Bumps [serde](https://github.com/serde-rs/serde) from 1.0.123 to 1.0.125.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.123...v1.0.125)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2021-03-23 04:24:16 +00:00
e574f4bdd0 Run tests also on windows
All checks were successful
continuous-integration/drone/push Build is passing
2021-03-22 13:41:50 +01:00
5c8e3fa35d Add badge test
All checks were successful
continuous-integration/drone/push Build is passing
2021-03-22 13:34:26 +01:00
4c0df1fa19 Add config so tests will work 2021-03-22 13:34:18 +01:00
f6b46a1e90 Add config so tests will work
All checks were successful
continuous-integration/drone/push Build is passing
2021-03-22 13:33:56 +01:00
5eb16ac38e Remove unused code
All checks were successful
continuous-integration/drone/push Build is passing
2021-03-22 12:52:24 +01:00
3486d44bc5 Restructure for testability and implement first tests 2021-03-22 12:46:10 +01:00
1311e724ce Bump version (v0.17.1)
All checks were successful
continuous-integration/drone/push Build is passing
continuous-integration/drone/tag Build is passing
2021-03-22 08:38:45 +01:00
5da9e74992 Merge pull request #248 from strdr4605/patch-1
fix: remove redundant param in md url
2021-03-22 08:37:53 +01:00
8dadc0ddea fix: remove redundant param in md url 2021-03-22 00:16:31 +02:00
cba235eadf Bump config from 0.10.1 to 0.11.0
Bumps [config](https://github.com/mehcode/config-rs) from 0.10.1 to 0.11.0.
- [Release notes](https://github.com/mehcode/config-rs/releases)
- [Changelog](https://github.com/mehcode/config-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/mehcode/config-rs/commits/0.11.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2021-03-18 04:18:27 +00:00
20 changed files with 894 additions and 729 deletions

View File

@ -69,7 +69,8 @@ jobs:
strategy:
matrix:
# add windows-latest when it is clear why tests are failing
os: [ubuntu-latest, macos-latest]
# os: [ubuntu-latest, macos-latest]
os: [ubuntu-latest]
steps:
- name: Checkout sources

1
.gitignore vendored
View File

@ -4,5 +4,4 @@ repos
cache
hoc.log
result
hoc.toml
.env

View File

@ -1,53 +0,0 @@
image: docker:19.03
services:
- docker:19.03-dind
stages:
- build
- release
variables:
DOCKER_HOST: tcp://docker:2375
DOCKER_DRIVER: overlay2
# DOCKER_TLS_CERTDIR: "/certs"
DOCKER_TLS_CERTDIR: ""
CONTAINER_BUILDER_IMAGE: $CI_REGISTRY_IMAGE:builder-latest
CONTAINER_TEST_IMAGE: $CI_REGISTRY_IMAGE:$CI_COMMIT_REF_SLUG
CONTAINER_RELEASE_IMAGE: $CI_REGISTRY_IMAGE:latest
before_script:
- export CONTAINER_TAG_IMAGE="$CI_REGISTRY_IMAGE:${CI_COMMIT_TAG:1}"
- docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
build:
stage: build
script:
- docker pull $CONTAINER_TEST_IMAGE || true
- docker pull $CONTAINER_RELEASE_IMAGE || true
- docker pull $CONTAINER_TAG_IMAGE || true
- docker pull $CONTAINER_BUILDER_IMAGE || true
- docker build --pull -t $CONTAINER_BUILDER_IMAGE --target builder .
- docker build --pull -t $CONTAINER_TEST_IMAGE .
- docker push $CONTAINER_BUILDER_IMAGE
- docker push $CONTAINER_TEST_IMAGE
release-image:
stage: release
script:
- docker pull $CONTAINER_TEST_IMAGE
- docker tag $CONTAINER_TEST_IMAGE $CONTAINER_RELEASE_IMAGE
- docker push $CONTAINER_RELEASE_IMAGE
only:
- master
release-tag:
stage: release
script:
- docker pull $CONTAINER_TEST_IMAGE
- docker tag $CONTAINER_TEST_IMAGE $CONTAINER_TAG_IMAGE
- docker push $CONTAINER_TAG_IMAGE
only:
- /^v\d+\.\d+\.\d+/
except:
- branch

229
Cargo.lock generated
View File

@ -80,7 +80,7 @@ dependencies = [
"pin-project 1.0.5",
"rand 0.7.3",
"regex",
"serde 1.0.123",
"serde 1.0.125",
"serde_json",
"serde_urlencoded",
"sha-1",
@ -108,7 +108,7 @@ dependencies = [
"http",
"log",
"regex",
"serde 1.0.123",
"serde 1.0.125",
]
[[package]]
@ -247,7 +247,7 @@ dependencies = [
"mime",
"pin-project 1.0.5",
"regex",
"serde 1.0.123",
"serde 1.0.125",
"serde_json",
"serde_urlencoded",
"socket2",
@ -300,6 +300,12 @@ dependencies = [
"winapi 0.3.9",
]
[[package]]
name = "anyhow"
version = "1.0.40"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "28b2cd92db5cbd74e8e5028f7e27dd7aa3090e89e4f2a197cc7c8dfb69c7063b"
[[package]]
name = "arrayvec"
version = "0.5.2"
@ -342,7 +348,7 @@ dependencies = [
"mime",
"percent-encoding",
"rand 0.7.3",
"serde 1.0.123",
"serde 1.0.125",
"serde_json",
"serde_urlencoded",
]
@ -396,6 +402,18 @@ version = "1.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cf1de2fe8c75bc145a2f577add951f8134889b4795d47466a54a5c846d691693"
[[package]]
name = "bitvec"
version = "0.19.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8942c8d352ae1838c9dda0b0ca2ab657696ef2232a20147cf1b30ae1a9cb4321"
dependencies = [
"funty",
"radium",
"tap",
"wyz",
]
[[package]]
name = "block-buffer"
version = "0.9.0"
@ -500,14 +518,14 @@ dependencies = [
[[package]]
name = "config"
version = "0.10.1"
version = "0.11.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "19b076e143e1d9538dde65da30f8481c2a6c44040edb8e02b9bf1351edb92ce3"
checksum = "1b1b9d958c2b1368a663f05538fc1b5975adce1e19f435acceae987aceeeb369"
dependencies = [
"lazy_static",
"nom",
"nom 5.1.2",
"rust-ini",
"serde 1.0.123",
"serde 1.0.125",
"serde-hjson",
"serde_json",
"toml",
@ -707,10 +725,16 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3dcaa9ae7725d12cdb85b3ad99a434db70b468c09ded17e012d86b5c1010f7a7"
[[package]]
name = "futures"
version = "0.3.13"
name = "funty"
version = "1.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7f55667319111d593ba876406af7c409c0ebb44dc4be6132a783ccf163ea14c1"
checksum = "fed34cd105917e91daa4da6b3728c47b068749d6a62c59811f06ed2ac71d9da7"
[[package]]
name = "futures"
version = "0.3.14"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a9d5813545e459ad3ca1bff9915e9ad7f1a47dc6a91b627ce321d5863b7dd253"
dependencies = [
"futures-channel",
"futures-core",
@ -723,9 +747,9 @@ dependencies = [
[[package]]
name = "futures-channel"
version = "0.3.13"
version = "0.3.14"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8c2dd2df839b57db9ab69c2c9d8f3e8c81984781937fe2807dc6dcf3b2ad2939"
checksum = "ce79c6a52a299137a6013061e0cf0e688fce5d7f1bc60125f520912fdb29ec25"
dependencies = [
"futures-core",
"futures-sink",
@ -733,15 +757,15 @@ dependencies = [
[[package]]
name = "futures-core"
version = "0.3.13"
version = "0.3.14"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "15496a72fabf0e62bdc3df11a59a3787429221dd0710ba8ef163d6f7a9112c94"
checksum = "098cd1c6dda6ca01650f1a37a794245eb73181d0d4d4e955e2f3c37db7af1815"
[[package]]
name = "futures-executor"
version = "0.3.13"
version = "0.3.14"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "891a4b7b96d84d5940084b2a37632dd65deeae662c114ceaa2c879629c9c0ad1"
checksum = "10f6cb7042eda00f0049b1d2080aa4b93442997ee507eb3828e8bd7577f94c9d"
dependencies = [
"futures-core",
"futures-task",
@ -750,15 +774,15 @@ dependencies = [
[[package]]
name = "futures-io"
version = "0.3.13"
version = "0.3.14"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d71c2c65c57704c32f5241c1223167c2c3294fd34ac020c807ddbe6db287ba59"
checksum = "365a1a1fb30ea1c03a830fdb2158f5236833ac81fa0ad12fe35b29cddc35cb04"
[[package]]
name = "futures-macro"
version = "0.3.13"
version = "0.3.14"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ea405816a5139fb39af82c2beb921d52143f556038378d6db21183a5c37fbfb7"
checksum = "668c6733a182cd7deb4f1de7ba3bf2120823835b3bcfbeacf7d2c4a773c1bb8b"
dependencies = [
"proc-macro-hack",
"proc-macro2",
@ -768,21 +792,21 @@ dependencies = [
[[package]]
name = "futures-sink"
version = "0.3.13"
version = "0.3.14"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "85754d98985841b7d4f5e8e6fbfa4a4ac847916893ec511a2917ccd8525b8bb3"
checksum = "5c5629433c555de3d82861a7a4e3794a4c40040390907cfbfd7143a92a426c23"
[[package]]
name = "futures-task"
version = "0.3.13"
version = "0.3.14"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "fa189ef211c15ee602667a6fcfe1c1fd9e07d42250d2156382820fba33c9df80"
checksum = "ba7aa51095076f3ba6d9a1f702f74bd05ec65f555d70d2033d55ba8d69f581bc"
[[package]]
name = "futures-util"
version = "0.3.13"
version = "0.3.14"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1812c7ab8aedf8d6f2701a43e1243acdbcc2b36ab26e2ad421eb99ac963d96d1"
checksum = "3c144ad54d60f23927f0a6b6d816e4271278b64f005ad65e4e35291d2de9c025"
dependencies = [
"futures-channel",
"futures-core",
@ -817,6 +841,16 @@ dependencies = [
"version_check",
]
[[package]]
name = "gethostname"
version = "0.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e692e296bfac1d2533ef168d0b60ff5897b8b70a4009276834014dd8924cc028"
dependencies = [
"libc",
"winapi 0.3.9",
]
[[package]]
name = "getrandom"
version = "0.1.16"
@ -918,7 +952,7 @@ dependencies = [
[[package]]
name = "hoc"
version = "0.17.0"
version = "0.17.1"
dependencies = [
"actix-rt",
"actix-web",
@ -933,13 +967,16 @@ dependencies = [
"openssl-probe",
"reqwest",
"ructe",
"serde 1.0.123",
"serde 1.0.125",
"serde_derive",
"serde_json",
"tempfile",
"tokio",
"tracing",
"tracing-actix-web",
"tracing-bunyan-formatter",
"tracing-futures",
"tracing-log",
"tracing-subscriber",
"vergen",
]
@ -1084,9 +1121,9 @@ checksum = "47be2f14c678be2fdcab04ab1171db51b2762ce6f0a8ee87c8dd4a04ed216135"
[[package]]
name = "itertools"
version = "0.9.0"
version = "0.10.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "284f18f85651fe11e8a991b2adb42cb078325c996ed026d994719efcfca1d54b"
checksum = "37d572918e350e82412fe766d24b15e6682fb2ed2bbe018280caa810397cb319"
dependencies = [
"either",
]
@ -1196,16 +1233,6 @@ dependencies = [
"vcpkg",
]
[[package]]
name = "linked-hash-map"
version = "0.3.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6d262045c5b87c0861b3f004610afd0e2c851e2908d08b6c870cbb9d5f494ecd"
dependencies = [
"serde 0.8.23",
"serde_test",
]
[[package]]
name = "linked-hash-map"
version = "0.5.4"
@ -1236,7 +1263,7 @@ version = "0.1.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "31e24f1ad8321ca0e8a1e0ac13f23cb668e6f5466c2c57319f6a5cf1cc8e3b1c"
dependencies = [
"linked-hash-map 0.5.4",
"linked-hash-map",
]
[[package]]
@ -1380,6 +1407,19 @@ dependencies = [
"version_check",
]
[[package]]
name = "nom"
version = "6.1.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e7413f999671bd4745a7b624bd370a569fb6bc574b23c83a3c5ed2e453f3d5e2"
dependencies = [
"bitvec",
"funty",
"lexical-core",
"memchr",
"version_check",
]
[[package]]
name = "num-integer"
version = "0.1.44"
@ -1645,6 +1685,12 @@ dependencies = [
"proc-macro2",
]
[[package]]
name = "radium"
version = "0.5.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "941ba9d78d8e2f7ce474c015eea4d9c6d25b6a3327f9832ee29a4de27f91bbb8"
[[package]]
name = "rand"
version = "0.7.3"
@ -1796,7 +1842,7 @@ dependencies = [
"native-tls",
"percent-encoding",
"pin-project-lite 0.2.4",
"serde 1.0.123",
"serde 1.0.125",
"serde_urlencoded",
"tokio",
"tokio-tls",
@ -1819,15 +1865,15 @@ dependencies = [
[[package]]
name = "ructe"
version = "0.13.0"
version = "0.13.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a3507c22423c8be907293f0aa684b08ac62efb20e0768639fdfbce833481fd30"
checksum = "5678e9cc1545f229509acb67cf34793802646f32c77e00bc470b518cfddea579"
dependencies = [
"base64 0.12.3",
"base64 0.13.0",
"bytecount",
"itertools",
"md5",
"nom",
"nom 6.1.2",
]
[[package]]
@ -1861,6 +1907,12 @@ dependencies = [
"owned_ttf_parser",
]
[[package]]
name = "rustversion"
version = "1.0.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cb5d2a036dc6d2d8fd16fde3498b04306e29bd193bf306a57427019b823d5acd"
[[package]]
name = "ryu"
version = "1.0.5"
@ -1929,9 +1981,9 @@ checksum = "9dad3f759919b92c3068c696c15c3d17238234498bbdcc80f2c469606f948ac8"
[[package]]
name = "serde"
version = "1.0.123"
version = "1.0.125"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "92d5161132722baa40d802cc70b15262b98258453e85e5d1d365c757c73869ae"
checksum = "558dc50e1a5a5fa7112ca2ce4effcb321b0300c0d4ccf0776a9f60cd89031171"
dependencies = [
"serde_derive",
]
@ -1943,7 +1995,6 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6a3a4e0ea8a88553209f6cc6cfe8724ecad22e1acf372793c27d995290fe74f8"
dependencies = [
"lazy_static",
"linked-hash-map 0.3.0",
"num-traits 0.1.43",
"regex",
"serde 0.8.23",
@ -1951,9 +2002,9 @@ dependencies = [
[[package]]
name = "serde_derive"
version = "1.0.123"
version = "1.0.125"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9391c295d64fc0abb2c556bad848f33cb8296276b1ad2677d1ae1ace4f258f31"
checksum = "b093b7a2bb58203b5da3056c05b4ec1fed827dcfdb37347a8841695263b3d06d"
dependencies = [
"proc-macro2",
"quote",
@ -1962,22 +2013,13 @@ dependencies = [
[[package]]
name = "serde_json"
version = "1.0.63"
version = "1.0.64"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "43535db9747a4ba938c0ce0a98cc631a46ebf943c9e1d604e091df6007620bf6"
checksum = "799e97dc9fdae36a5c8b8f2cae9ce2ee9fdce2058c57a93e6099d919fd982f79"
dependencies = [
"itoa",
"ryu",
"serde 1.0.123",
]
[[package]]
name = "serde_test"
version = "0.8.23"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "110b3dbdf8607ec493c22d5d947753282f3bae73c0f56d322af1e8c78e4c23d5"
dependencies = [
"serde 0.8.23",
"serde 1.0.125",
]
[[package]]
@ -1989,7 +2031,7 @@ dependencies = [
"form_urlencoded",
"itoa",
"ryu",
"serde 1.0.123",
"serde 1.0.125",
]
[[package]]
@ -2089,7 +2131,7 @@ checksum = "c87a60a40fccc84bef0652345bbbbbe20a605bf5d0ce81719fc476f5c03b50ef"
dependencies = [
"proc-macro2",
"quote",
"serde 1.0.123",
"serde 1.0.125",
"serde_derive",
"syn",
]
@ -2103,7 +2145,7 @@ dependencies = [
"base-x",
"proc-macro2",
"quote",
"serde 1.0.123",
"serde 1.0.125",
"serde_derive",
"serde_json",
"sha1",
@ -2127,6 +2169,12 @@ dependencies = [
"unicode-xid",
]
[[package]]
name = "tap"
version = "1.0.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "55937e1799185b12863d447f42597ed69d9928686b8d88a1df17376a097d8369"
[[package]]
name = "tempfile"
version = "3.2.0"
@ -2293,7 +2341,7 @@ version = "0.5.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a31142970826733df8241ef35dc040ef98c679ab14d7c3e54d827099b3acecaa"
dependencies = [
"serde 1.0.123",
"serde 1.0.125",
]
[[package]]
@ -2339,6 +2387,23 @@ dependencies = [
"syn",
]
[[package]]
name = "tracing-bunyan-formatter"
version = "0.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "05e6b1b6c038da11d8d704f93b3974b0394824cdcc8c818c48255183cf315fd5"
dependencies = [
"chrono",
"gethostname",
"log",
"serde 1.0.125",
"serde_json",
"tracing",
"tracing-core",
"tracing-log",
"tracing-subscriber",
]
[[package]]
name = "tracing-core"
version = "0.1.17"
@ -2375,22 +2440,22 @@ version = "0.1.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "fb65ea441fbb84f9f6748fd496cf7f63ec9af5bca94dd86456978d055e8eb28b"
dependencies = [
"serde 1.0.123",
"serde 1.0.125",
"tracing-core",
]
[[package]]
name = "tracing-subscriber"
version = "0.2.16"
version = "0.2.17"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8ab8966ac3ca27126141f7999361cc97dd6fb4b71da04c02044fa9045d98bb96"
checksum = "705096c6f83bf68ea5d357a6aa01829ddbdac531b357b45abeca842938085baa"
dependencies = [
"ansi_term",
"chrono",
"lazy_static",
"matchers",
"regex",
"serde 1.0.123",
"serde 1.0.125",
"serde_json",
"sharded-slab",
"smallvec",
@ -2527,17 +2592,17 @@ checksum = "b00bca6106a5e23f3eee943593759b7fcddb00554332e856d990c893966879fb"
[[package]]
name = "vergen"
version = "4.1.0"
version = "5.1.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7559dbf502657d16ddfdee53eea0827fc40ea7e3a53bc8b92d7317417cf9420e"
checksum = "05b12ae3b7d7a942d4698d79e20513c99fb3b6b7b0efc1b1f1fb0aafa31de939"
dependencies = [
"bitflags",
"anyhow",
"chrono",
"enum-iterator",
"getset",
"git2",
"serde 1.0.123",
"serde_derive",
"rustversion",
"thiserror",
]
[[package]]
@ -2575,7 +2640,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "55c0f7123de74f0dab9b7d00fd614e7b19349cd1e2f5252bbe9b1754b59433be"
dependencies = [
"cfg-if 1.0.0",
"serde 1.0.123",
"serde 1.0.125",
"serde_json",
"wasm-bindgen-macro",
]
@ -2714,11 +2779,17 @@ dependencies = [
"winapi-build",
]
[[package]]
name = "wyz"
version = "0.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "85e60b0d1b5f99db2556934e21937020776a5d31520bf169e851ac44e6420214"
[[package]]
name = "yaml-rust"
version = "0.4.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "56c1936c4cc7a1c9ab21a1ebb602eb942ba868cbd44a99cb7cdc5892335e1c85"
dependencies = [
"linked-hash-map 0.5.4",
"linked-hash-map",
]

View File

@ -1,34 +1,45 @@
[package]
name = "hoc"
version = "0.17.0"
version = "0.17.1"
authors = ["Valentin Brandl <vbrandl@riseup.net>"]
edition = "2018"
build = "build.rs"
[lib]
path = "src/lib.rs"
[[bin]]
path = "src/main.rs"
name = "hoc"
[dependencies]
actix-rt = "1.1.1"
actix-web = "3.3.2"
badge = "0.3.0"
bytes = "1.0.1"
config = { version = "0.10.1", features = ["toml"] }
config = { version = "0.11.0", features = ["toml"] }
dotenv = "0.15.0"
futures = "0.3.13"
futures = "0.3.14"
git2 = "0.13.17"
lazy_static = "1.4.0"
number_prefix = "0.4.0"
openssl-probe = "0.1.2"
reqwest = "0.10.10"
serde = "1.0.123"
serde = "1.0.125"
serde_derive = "1.0.123"
serde_json = "1.0.63"
serde_json = "1.0.64"
tracing = "0.1.25"
tracing-actix-web = "0.2.1"
tracing-bunyan-formatter = "0.2.0"
tracing-futures = "0.2.5"
tracing-subscriber = "0.2.16"
tracing-log = "0.1.2"
tracing-subscriber = { version = "0.2.17", features = ["registry", "env-filter"] }
[build-dependencies]
ructe = "0.13.0"
vergen = { version = "4.1.0", default-features = false, features = ["git"] }
ructe = "0.13.2"
vergen = { version = "5.1.1", default-features = false, features = ["git"] }
[dev-dependencies]
ructe = "0.13.2"
tempfile = "3.2.0"
tokio = "0.2.25"

View File

@ -2,11 +2,12 @@ extern crate ructe;
extern crate vergen;
use ructe::Ructe;
use vergen::{gen, ConstantsFlags};
use vergen::{vergen, Config, ShaKind};
fn main() {
let flags = ConstantsFlags::SHA_SHORT;
gen(flags).expect("Unable to generate the cargo keys!");
let mut config = Config::default();
*config.git_mut().sha_kind_mut() = ShaKind::Short;
vergen(config).expect("Unable to generate static repo info");
Ructe::from_env()
.expect("ructe")
.compile_templates("templates")

15
hoc.toml Normal file
View File

@ -0,0 +1,15 @@
# every parameter can also be set (or overwritten) by passing an environment
# variable namend `HOC_<PARAMETERNAME>`, e.g.
# `HOC_BASE_URL='https://hitsofcode.com' ./hoc`
# these config parameters have default values and must not explicitly be set
repodir = "./repos"
cachedir = "./cache"
port = 8080
host = "0.0.0.0"
workers = 4
# these parameters don't have default values and must be set
# this should be the public base URL of the service, e.g. `https://hitsofcode.com`
base_url = "http://0.0.0.0:8080"

View File

@ -17,16 +17,13 @@ pub struct Settings {
pub workers: usize,
}
pub(crate) fn init() {
dotenv::dotenv().ok();
std::env::set_var("RUST_LOG", "actix_web=info,hoc=info");
openssl_probe::init_ssl_cert_env_vars();
tracing_subscriber::fmt().init();
}
impl Settings {
#[deprecated]
pub fn new() -> Result<Self, ConfigError> {
Self::load()
}
pub fn load() -> Result<Self, ConfigError> {
let mut config = Config::new();
config
.merge(File::with_name("hoc.toml").required(false))?

View File

@ -1,9 +1,6 @@
use crate::{
statics::{REPO_COUNT, VERSION_INFO},
templates,
};
use actix_web::{HttpResponse, ResponseError};
use std::{fmt, sync::atomic::Ordering};
use crate::{statics::VERSION_INFO, templates};
use actix_web::{http::StatusCode, HttpResponse, ResponseError};
use std::fmt;
pub(crate) type Result<T> = std::result::Result<T, Error>;
@ -35,21 +32,22 @@ impl fmt::Display for Error {
}
impl ResponseError for Error {
fn status_code(&self) -> StatusCode {
match self {
Error::BranchNotFound => StatusCode::NOT_FOUND,
_ => StatusCode::INTERNAL_SERVER_ERROR,
}
}
fn error_response(&self) -> HttpResponse {
let mut buf = Vec::new();
match self {
Error::BranchNotFound => {
templates::p404_no_master(
&mut buf,
VERSION_INFO,
REPO_COUNT.load(Ordering::Relaxed),
)
.unwrap();
templates::p404_no_master(&mut buf, VERSION_INFO, 0).unwrap();
HttpResponse::NotFound().content_type("text/html").body(buf)
}
_ => {
templates::p500(&mut buf, VERSION_INFO, REPO_COUNT.load(Ordering::Relaxed))
.unwrap();
templates::p500(&mut buf, VERSION_INFO, 0).unwrap();
HttpResponse::InternalServerError()
.content_type("text/html")
.body(buf)

525
src/lib.rs Normal file
View File

@ -0,0 +1,525 @@
#![type_length_limit = "2257138"]
#[macro_use]
extern crate actix_web;
#[macro_use]
extern crate lazy_static;
#[macro_use]
extern crate serde_derive;
#[macro_use]
extern crate tracing;
mod cache;
pub mod config;
mod count;
mod error;
mod service;
mod statics;
pub mod telemetry;
mod template;
use crate::{
cache::CacheState,
config::Settings,
error::{Error, Result},
service::{Bitbucket, FormService, GitHub, Gitlab, Service},
statics::{CLIENT, CSS, FAVICON, VERSION_INFO},
template::RepoInfo,
};
use actix_web::{
dev::Server,
http::header::{CacheControl, CacheDirective, Expires, LOCATION},
middleware::{self, normalize::TrailingSlash},
web, App, HttpResponse, HttpServer, Responder,
};
use badge::{Badge, BadgeOptions};
use git2::{BranchType, Repository};
use number_prefix::NumberPrefix;
use std::{
borrow::Cow,
fs::create_dir_all,
io,
net::TcpListener,
path::Path,
process::Command,
sync::atomic::AtomicUsize,
sync::atomic::Ordering,
time::{Duration, SystemTime},
};
use tracing::Instrument;
include!(concat!(env!("OUT_DIR"), "/templates.rs"));
#[derive(Deserialize, Serialize)]
struct GeneratorForm<'a> {
service: FormService,
user: Cow<'a, str>,
repo: Cow<'a, str>,
}
#[derive(Debug)]
pub(crate) struct State {
settings: Settings,
}
impl State {
fn repos(&self) -> String {
self.settings.repodir.display().to_string()
}
fn cache(&self) -> String {
self.settings.cachedir.display().to_string()
}
}
#[derive(Serialize)]
struct JsonResponse<'a> {
head: &'a str,
branch: &'a str,
count: u64,
commits: u64,
}
#[derive(Deserialize, Debug)]
struct BranchQuery {
branch: Option<String>,
}
fn pull(path: impl AsRef<Path>) -> Result<()> {
let repo = Repository::open_bare(path)?;
let mut origin = repo.find_remote("origin")?;
origin.fetch(&["refs/heads/*:refs/heads/*"], None, None)?;
Ok(())
}
fn hoc(repo: &str, repo_dir: &str, cache_dir: &str, branch: &str) -> Result<(u64, String, u64)> {
let repo_dir = format!("{}/{}", repo_dir, repo);
let cache_dir = format!("{}/{}.json", cache_dir, repo);
let cache_dir = Path::new(&cache_dir);
let repo = Repository::open_bare(&repo_dir)?;
// TODO: do better...
let head = repo
.find_branch(branch, BranchType::Local)
.map_err(|_| Error::BranchNotFound)?
.into_reference();
let head = format!("{}", head.target().ok_or(Error::BranchNotFound)?);
let mut arg_commit_count = vec!["rev-list".to_string(), "--count".to_string()];
let mut arg = vec![
"log".to_string(),
"--pretty=tformat:".to_string(),
"--numstat".to_string(),
"--ignore-space-change".to_string(),
"--ignore-all-space".to_string(),
"--ignore-submodules".to_string(),
"--no-color".to_string(),
"--find-copies-harder".to_string(),
"-M".to_string(),
"--diff-filter=ACDM".to_string(),
];
let cache = CacheState::read_from_file(&cache_dir, branch, &head)?;
match &cache {
CacheState::Current { count, commits, .. } => {
info!("Using cache");
return Ok((*count, head, *commits));
}
CacheState::Old { head, .. } => {
info!("Updating cache");
arg.push(format!("{}..{}", head, branch));
arg_commit_count.push(format!("{}..{}", head, branch));
}
CacheState::No | CacheState::NoneForBranch(..) => {
info!("Creating cache");
arg.push(branch.to_string());
arg_commit_count.push(branch.to_string());
}
};
arg.push("--".to_string());
arg.push(".".to_string());
let output = Command::new("git")
.args(&arg)
.current_dir(&repo_dir)
.output()?
.stdout;
let output = String::from_utf8_lossy(&output);
let output_commits = Command::new("git")
.args(&arg_commit_count)
.current_dir(&repo_dir)
.output()?
.stdout;
let output_commits = String::from_utf8_lossy(&output_commits);
let commits: u64 = output_commits.trim().parse()?;
let count: u64 = output
.lines()
.map(|s| {
s.split_whitespace()
.take(2)
.map(str::parse::<u64>)
.filter_map(std::result::Result::ok)
.sum::<u64>()
})
.sum();
let cache = cache.calculate_new_cache(count, commits, (&head).into(), branch);
cache.write_to_file(cache_dir)?;
Ok((count, head, commits))
}
async fn remote_exists(url: &str) -> Result<bool> {
let resp = CLIENT.head(url).send().await?;
Ok(resp.status() == reqwest::StatusCode::OK)
}
enum HocResult {
Hoc {
hoc: u64,
commits: u64,
hoc_pretty: String,
head: String,
url: String,
repo: String,
service_path: String,
},
NotFound,
}
async fn delete_repo_and_cache<T>(
state: web::Data<State>,
repo_count: web::Data<AtomicUsize>,
data: web::Path<(String, String)>,
) -> Result<impl Responder>
where
T: Service,
{
let data = data.into_inner();
let span = info_span!(
"deleting repository and cache",
service = T::domain(),
user = data.0.as_str(),
repo = data.1.as_str()
);
let future = async {
let repo = format!(
"{}/{}/{}",
T::domain(),
data.0.to_lowercase(),
data.1.to_lowercase()
);
info!("Deleting cache and repository");
let cache_dir = format!("{}/{}.json", &state.cache(), repo);
let repo_dir = format!("{}/{}", &state.repos(), repo);
std::fs::remove_file(&cache_dir).or_else(|e| {
if e.kind() == io::ErrorKind::NotFound {
Ok(())
} else {
Err(e)
}
})?;
std::fs::remove_dir_all(&repo_dir).or_else(|e| {
if e.kind() == io::ErrorKind::NotFound {
Ok(())
} else {
Err(e)
}
})?;
repo_count.fetch_sub(1, Ordering::Relaxed);
Ok(HttpResponse::TemporaryRedirect()
.header(
LOCATION,
format!("/{}/{}/{}/view", T::url_path(), data.0, data.1),
)
.finish())
};
future.instrument(span).await
}
async fn handle_hoc_request<T, F>(
state: web::Data<State>,
repo_count: web::Data<AtomicUsize>,
data: web::Path<(String, String)>,
branch: &str,
mapper: F,
) -> Result<HttpResponse>
where
T: Service,
F: FnOnce(HocResult) -> Result<HttpResponse>,
{
let data = data.into_inner();
let span = info_span!(
"handling hoc calculation",
service = T::domain(),
user = data.0.as_str(),
repo = data.1.as_str(),
branch
);
let future = async {
let repo = format!("{}/{}", data.0.to_lowercase(), data.1.to_lowercase());
let service_path = format!("{}/{}", T::url_path(), repo);
let service_url = format!("{}/{}", T::domain(), repo);
let path = format!("{}/{}", state.repos(), service_url);
let url = format!("https://{}", service_url);
let remote_exists = remote_exists(&url).await?;
let file = Path::new(&path);
if !file.exists() {
if !remote_exists {
warn!("Repository does not exist");
return mapper(HocResult::NotFound);
}
info!("Cloning for the first time");
create_dir_all(file)?;
let repo = Repository::init_bare(file)?;
repo.remote_add_fetch("origin", "refs/heads/*:refs/heads/*")?;
repo.remote_set_url("origin", &url)?;
repo_count.fetch_add(1, Ordering::Relaxed);
}
pull(&path)?;
let (hoc, head, commits) = hoc(&service_url, &state.repos(), &state.cache(), branch)?;
let hoc_pretty = match NumberPrefix::decimal(hoc as f64) {
NumberPrefix::Standalone(hoc) => hoc.to_string(),
NumberPrefix::Prefixed(prefix, hoc) => format!("{:.1}{}", hoc, prefix),
};
let res = HocResult::Hoc {
hoc,
commits,
hoc_pretty,
head,
url,
repo,
service_path,
};
mapper(res)
};
future.instrument(span).await
}
pub(crate) async fn json_hoc<T: Service>(
state: web::Data<State>,
repo_count: web::Data<AtomicUsize>,
data: web::Path<(String, String)>,
branch: web::Query<BranchQuery>,
) -> Result<HttpResponse> {
let branch = branch.branch.as_deref().unwrap_or("master");
let rc_clone = repo_count.clone();
let mapper = move |r| match r {
HocResult::NotFound => p404(rc_clone),
HocResult::Hoc {
hoc, head, commits, ..
} => Ok(HttpResponse::Ok().json(JsonResponse {
branch,
head: &head,
count: hoc,
commits,
})),
};
handle_hoc_request::<T, _>(state, repo_count, data, branch, mapper).await
}
fn no_cache_response(body: Vec<u8>) -> HttpResponse {
let expiration = SystemTime::now() + Duration::from_secs(30);
HttpResponse::Ok()
.content_type("image/svg+xml")
.set(Expires(expiration.into()))
.set(CacheControl(vec![
CacheDirective::MaxAge(0u32),
CacheDirective::MustRevalidate,
CacheDirective::NoCache,
CacheDirective::NoStore,
]))
.body(body)
}
pub(crate) async fn calculate_hoc<T: Service>(
state: web::Data<State>,
repo_count: web::Data<AtomicUsize>,
data: web::Path<(String, String)>,
branch: web::Query<BranchQuery>,
) -> HttpResponse {
let rc_clone = repo_count.clone();
let mapper = move |r| match r {
HocResult::NotFound => p404(rc_clone),
HocResult::Hoc { hoc_pretty, .. } => {
let badge_opt = BadgeOptions {
subject: "Hits-of-Code".to_string(),
color: "#007ec6".to_string(),
status: hoc_pretty,
};
let badge = Badge::new(badge_opt)?;
// TODO: remove clone
let body = badge.to_svg().as_bytes().to_vec();
Ok(no_cache_response(body))
}
};
let branch = branch.branch.as_deref().unwrap_or("master");
let error_badge = |_| {
let error_badge = Badge::new(BadgeOptions {
subject: "Hits-of-Code".to_string(),
color: "#ff0000".to_string(),
status: "error".to_string(),
})
.unwrap();
let body = error_badge.to_svg().as_bytes().to_vec();
no_cache_response(body)
};
handle_hoc_request::<T, _>(state, repo_count, data, branch, mapper)
.await
.unwrap_or_else(error_badge)
}
async fn overview<T: Service>(
state: web::Data<State>,
repo_count: web::Data<AtomicUsize>,
data: web::Path<(String, String)>,
branch: web::Query<BranchQuery>,
) -> Result<HttpResponse> {
let branch = branch.branch.as_deref().unwrap_or("master");
let base_url = state.settings.base_url.clone();
let rc_clone = repo_count.clone();
let mapper = move |r| match r {
HocResult::NotFound => p404(rc_clone),
HocResult::Hoc {
hoc,
commits,
hoc_pretty,
url,
head,
repo,
service_path,
} => {
let mut buf = Vec::new();
let repo_info = RepoInfo {
commit_url: &T::commit_url(&repo, &head),
commits,
base_url: &base_url,
head: &head,
hoc,
hoc_pretty: &hoc_pretty,
path: &service_path,
url: &url,
branch,
};
templates::overview(
&mut buf,
VERSION_INFO,
rc_clone.load(Ordering::Relaxed),
repo_info,
)?;
Ok(HttpResponse::Ok().content_type("text/html").body(buf))
}
};
handle_hoc_request::<T, _>(state, repo_count, data, branch, mapper).await
}
#[get("/health_check")]
async fn health_check() -> HttpResponse {
HttpResponse::Ok().finish()
}
#[get("/")]
async fn index(
state: web::Data<State>,
repo_count: web::Data<AtomicUsize>,
) -> Result<HttpResponse> {
let mut buf = Vec::new();
templates::index(
&mut buf,
VERSION_INFO,
repo_count.load(Ordering::Relaxed),
&state.settings.base_url,
)?;
Ok(HttpResponse::Ok().content_type("text/html").body(buf))
}
#[post("/generate")]
async fn generate(
params: web::Form<GeneratorForm<'_>>,
state: web::Data<State>,
repo_count: web::Data<AtomicUsize>,
) -> Result<HttpResponse> {
let repo = format!("{}/{}", params.user, params.repo);
let mut buf = Vec::new();
templates::generate(
&mut buf,
VERSION_INFO,
repo_count.load(Ordering::Relaxed),
&state.settings.base_url,
params.service.url(),
params.service.service(),
&repo,
)?;
Ok(HttpResponse::Ok().content_type("text/html").body(buf))
}
fn p404(repo_count: web::Data<AtomicUsize>) -> Result<HttpResponse> {
let mut buf = Vec::new();
templates::p404(&mut buf, VERSION_INFO, repo_count.load(Ordering::Relaxed))?;
Ok(HttpResponse::NotFound().content_type("text/html").body(buf))
}
async fn async_p404(repo_count: web::Data<AtomicUsize>) -> Result<HttpResponse> {
p404(repo_count)
}
fn css() -> HttpResponse {
HttpResponse::Ok().content_type("text/css").body(CSS)
}
fn favicon32() -> HttpResponse {
HttpResponse::Ok().content_type("image/png").body(FAVICON)
}
async fn start_server(listener: TcpListener, settings: Settings) -> std::io::Result<Server> {
let workers = settings.workers;
let repo_count =
// TODO: errorhandling
web::Data::new(AtomicUsize::new(count::count_repositories(&settings.repodir).unwrap()));
let state = web::Data::new(State { settings });
Ok(HttpServer::new(move || {
App::new()
.app_data(state.clone())
.app_data(repo_count.clone())
.wrap(tracing_actix_web::TracingLogger)
.wrap(middleware::NormalizePath::new(TrailingSlash::Trim))
.service(index)
.service(health_check)
.service(web::resource("/tacit-css.min.css").route(web::get().to(css)))
.service(web::resource("/favicon.ico").route(web::get().to(favicon32)))
.service(generate)
.service(web::resource("/github/{user}/{repo}").to(calculate_hoc::<GitHub>))
.service(web::resource("/gitlab/{user}/{repo}").to(calculate_hoc::<Gitlab>))
.service(web::resource("/bitbucket/{user}/{repo}").to(calculate_hoc::<Bitbucket>))
.service(
web::resource("/github/{user}/{repo}/delete")
.route(web::post().to(delete_repo_and_cache::<GitHub>)),
)
.service(
web::resource("/gitlab/{user}/{repo}/delete")
.route(web::post().to(delete_repo_and_cache::<Gitlab>)),
)
.service(
web::resource("/bitbucket/{user}/{repo}/delete")
.route(web::post().to(delete_repo_and_cache::<Bitbucket>)),
)
.service(web::resource("/github/{user}/{repo}/json").to(json_hoc::<GitHub>))
.service(web::resource("/gitlab/{user}/{repo}/json").to(json_hoc::<Gitlab>))
.service(web::resource("/bitbucket/{user}/{repo}/json").to(json_hoc::<Bitbucket>))
.service(web::resource("/view/github/{user}/{repo}").to(overview::<GitHub>))
.service(web::resource("/view/gitlab/{user}/{repo}").to(overview::<Gitlab>))
.service(web::resource("/view/bitbucket/{user}/{repo}").to(overview::<Bitbucket>))
.service(web::resource("/github/{user}/{repo}/view").to(overview::<GitHub>))
.service(web::resource("/gitlab/{user}/{repo}/view").to(overview::<Gitlab>))
.service(web::resource("/bitbucket/{user}/{repo}/view").to(overview::<Bitbucket>))
.default_service(web::resource("").route(web::get().to(async_p404)))
})
.workers(workers)
.listen(listener)?
.run())
}
pub async fn run(listener: TcpListener, settings: Settings) -> std::io::Result<Server> {
let span = info_span!("hoc", version = env!("CARGO_PKG_VERSION"));
let _ = span.enter();
start_server(listener, settings).instrument(span).await
}

View File

@ -1,495 +1,26 @@
#![type_length_limit = "2257138"]
use hoc::{config::Settings, telemetry};
#[macro_use]
extern crate actix_web;
#[macro_use]
extern crate lazy_static;
#[macro_use]
extern crate serde_derive;
#[macro_use]
extern crate tracing;
use std::net::TcpListener;
mod cache;
mod config;
mod count;
mod error;
mod service;
mod statics;
mod template;
fn init() {
dotenv::dotenv().ok();
openssl_probe::init_ssl_cert_env_vars();
#[cfg(test)]
mod tests;
use crate::{
cache::CacheState,
error::{Error, Result},
service::{Bitbucket, FormService, GitHub, Gitlab, Service},
statics::{CLIENT, CSS, FAVICON, OPT, REPO_COUNT, VERSION_INFO},
template::RepoInfo,
};
use actix_web::{
http::header::{CacheControl, CacheDirective, Expires, LOCATION},
middleware::{self, normalize::TrailingSlash},
web, App, HttpResponse, HttpServer, Responder,
};
use badge::{Badge, BadgeOptions};
use git2::{BranchType, Repository};
use number_prefix::NumberPrefix;
use std::{
borrow::Cow,
fs::create_dir_all,
io,
path::Path,
process::Command,
sync::atomic::Ordering,
sync::Arc,
time::{Duration, SystemTime},
};
use tracing::Instrument;
include!(concat!(env!("OUT_DIR"), "/templates.rs"));
#[derive(Deserialize, Serialize)]
struct GeneratorForm<'a> {
service: FormService,
user: Cow<'a, str>,
repo: Cow<'a, str>,
}
#[derive(Debug)]
pub(crate) struct State {
repos: String,
cache: String,
}
#[derive(Serialize)]
struct JsonResponse<'a> {
head: &'a str,
branch: &'a str,
count: u64,
commits: u64,
}
#[derive(Deserialize, Debug)]
struct BranchQuery {
branch: Option<String>,
}
fn pull(path: impl AsRef<Path>) -> Result<()> {
let repo = Repository::open_bare(path)?;
let mut origin = repo.find_remote("origin")?;
origin.fetch(&["refs/heads/*:refs/heads/*"], None, None)?;
Ok(())
}
fn hoc(repo: &str, repo_dir: &str, cache_dir: &str, branch: &str) -> Result<(u64, String, u64)> {
let repo_dir = format!("{}/{}", repo_dir, repo);
let cache_dir = format!("{}/{}.json", cache_dir, repo);
let cache_dir = Path::new(&cache_dir);
let repo = Repository::open_bare(&repo_dir)?;
// TODO: do better...
let head = repo
.find_branch(branch, BranchType::Local)
.map_err(|_| Error::BranchNotFound)?
.into_reference();
let head = format!("{}", head.target().ok_or(Error::BranchNotFound)?);
let mut arg_commit_count = vec!["rev-list".to_string(), "--count".to_string()];
let mut arg = vec![
"log".to_string(),
"--pretty=tformat:".to_string(),
"--numstat".to_string(),
"--ignore-space-change".to_string(),
"--ignore-all-space".to_string(),
"--ignore-submodules".to_string(),
"--no-color".to_string(),
"--find-copies-harder".to_string(),
"-M".to_string(),
"--diff-filter=ACDM".to_string(),
];
let cache = CacheState::read_from_file(&cache_dir, branch, &head)?;
match &cache {
CacheState::Current { count, commits, .. } => {
info!("Using cache");
return Ok((*count, head, *commits));
}
CacheState::Old { head, .. } => {
info!("Updating cache");
arg.push(format!("{}..{}", head, branch));
arg_commit_count.push(format!("{}..{}", head, branch));
}
CacheState::No | CacheState::NoneForBranch(..) => {
info!("Creating cache");
arg.push(branch.to_string());
arg_commit_count.push(branch.to_string());
}
};
arg.push("--".to_string());
arg.push(".".to_string());
let output = Command::new("git")
.args(&arg)
.current_dir(&repo_dir)
.output()?
.stdout;
let output = String::from_utf8_lossy(&output);
let output_commits = Command::new("git")
.args(&arg_commit_count)
.current_dir(&repo_dir)
.output()?
.stdout;
let output_commits = String::from_utf8_lossy(&output_commits);
let commits: u64 = output_commits.trim().parse()?;
let count: u64 = output
.lines()
.map(|s| {
s.split_whitespace()
.take(2)
.map(str::parse::<u64>)
.filter_map(std::result::Result::ok)
.sum::<u64>()
})
.sum();
let cache = cache.calculate_new_cache(count, commits, (&head).into(), branch);
cache.write_to_file(cache_dir)?;
Ok((count, head, commits))
}
async fn remote_exists(url: &str) -> Result<bool> {
let resp = CLIENT.head(url).send().await?;
Ok(resp.status() == reqwest::StatusCode::OK)
}
enum HocResult {
Hoc {
hoc: u64,
commits: u64,
hoc_pretty: String,
head: String,
url: String,
repo: String,
service_path: String,
},
NotFound,
}
async fn delete_repo_and_cache<T>(
state: web::Data<Arc<State>>,
data: web::Path<(String, String)>,
) -> Result<impl Responder>
where
T: Service,
{
let data = data.into_inner();
let span = info_span!(
"deleting repository and cache",
service = T::domain(),
user = data.0.as_str(),
repo = data.1.as_str()
);
let future = async {
let repo = format!(
"{}/{}/{}",
T::domain(),
data.0.to_lowercase(),
data.1.to_lowercase()
);
info!("Deleting cache and repository");
let cache_dir = format!("{}/{}.json", &state.cache, repo);
let repo_dir = format!("{}/{}", &state.repos, repo);
std::fs::remove_file(&cache_dir).or_else(|e| {
if e.kind() == io::ErrorKind::NotFound {
Ok(())
} else {
Err(e)
}
})?;
std::fs::remove_dir_all(&repo_dir).or_else(|e| {
if e.kind() == io::ErrorKind::NotFound {
Ok(())
} else {
Err(e)
}
})?;
REPO_COUNT.fetch_sub(1, Ordering::Relaxed);
Ok(HttpResponse::TemporaryRedirect()
.header(
LOCATION,
format!("/{}/{}/{}/view", T::url_path(), data.0, data.1),
)
.finish())
};
future.instrument(span).await
}
async fn handle_hoc_request<T, F>(
state: web::Data<Arc<State>>,
data: web::Path<(String, String)>,
branch: &str,
mapper: F,
) -> Result<HttpResponse>
where
T: Service,
F: Fn(HocResult) -> Result<HttpResponse>,
{
let data = data.into_inner();
let span = info_span!(
"handling hoc calculation",
service = T::domain(),
user = data.0.as_str(),
repo = data.1.as_str(),
branch
);
let future = async {
let repo = format!("{}/{}", data.0.to_lowercase(), data.1.to_lowercase());
let service_path = format!("{}/{}", T::url_path(), repo);
let service_url = format!("{}/{}", T::domain(), repo);
let path = format!("{}/{}", state.repos, service_url);
let url = format!("https://{}", service_url);
let remote_exists = remote_exists(&url).await?;
let file = Path::new(&path);
if !file.exists() {
if !remote_exists {
warn!("Repository does not exist");
return mapper(HocResult::NotFound);
}
info!("Cloning for the first time");
create_dir_all(file)?;
let repo = Repository::init_bare(file)?;
repo.remote_add_fetch("origin", "refs/heads/*:refs/heads/*")?;
repo.remote_set_url("origin", &url)?;
REPO_COUNT.fetch_add(1, Ordering::Relaxed);
}
pull(&path)?;
let (hoc, head, commits) = hoc(&service_url, &state.repos, &state.cache, branch)?;
let hoc_pretty = match NumberPrefix::decimal(hoc as f64) {
NumberPrefix::Standalone(hoc) => hoc.to_string(),
NumberPrefix::Prefixed(prefix, hoc) => format!("{:.1}{}", hoc, prefix),
};
let res = HocResult::Hoc {
hoc,
commits,
hoc_pretty,
head,
url,
repo,
service_path,
};
mapper(res)
};
future.instrument(span).await
}
pub(crate) async fn json_hoc<T: Service>(
state: web::Data<Arc<State>>,
data: web::Path<(String, String)>,
branch: web::Query<BranchQuery>,
) -> Result<HttpResponse> {
let branch = branch.branch.as_deref().unwrap_or("master");
let mapper = |r| match r {
HocResult::NotFound => p404(),
HocResult::Hoc {
hoc, head, commits, ..
} => Ok(HttpResponse::Ok().json(JsonResponse {
branch,
head: &head,
count: hoc,
commits,
})),
};
handle_hoc_request::<T, _>(state, data, branch, mapper).await
}
fn no_cache_response(body: Vec<u8>) -> HttpResponse {
let expiration = SystemTime::now() + Duration::from_secs(30);
HttpResponse::Ok()
.content_type("image/svg+xml")
.set(Expires(expiration.into()))
.set(CacheControl(vec![
CacheDirective::MaxAge(0u32),
CacheDirective::MustRevalidate,
CacheDirective::NoCache,
CacheDirective::NoStore,
]))
.body(body)
}
pub(crate) async fn calculate_hoc<T: Service>(
state: web::Data<Arc<State>>,
data: web::Path<(String, String)>,
branch: web::Query<BranchQuery>,
) -> HttpResponse {
let mapper = move |r| match r {
HocResult::NotFound => p404(),
HocResult::Hoc { hoc_pretty, .. } => {
let badge_opt = BadgeOptions {
subject: "Hits-of-Code".to_string(),
color: "#007ec6".to_string(),
status: hoc_pretty,
};
let badge = Badge::new(badge_opt)?;
// TODO: remove clone
let body = badge.to_svg().as_bytes().to_vec();
Ok(no_cache_response(body))
}
};
let branch = branch.branch.as_deref().unwrap_or("master");
let error_badge = |_| {
let error_badge = Badge::new(BadgeOptions {
subject: "Hits-of-Code".to_string(),
color: "#ff0000".to_string(),
status: "error".to_string(),
})
.unwrap();
let body = error_badge.to_svg().as_bytes().to_vec();
no_cache_response(body)
};
handle_hoc_request::<T, _>(state, data, branch, mapper)
.await
.unwrap_or_else(error_badge)
}
async fn overview<T: Service>(
state: web::Data<Arc<State>>,
data: web::Path<(String, String)>,
branch: web::Query<BranchQuery>,
) -> Result<HttpResponse> {
let branch = branch.branch.as_deref().unwrap_or("master");
let mapper = |r| match r {
HocResult::NotFound => p404(),
HocResult::Hoc {
hoc,
commits,
hoc_pretty,
url,
head,
repo,
service_path,
} => {
let mut buf = Vec::new();
let repo_info = RepoInfo {
commit_url: &T::commit_url(&repo, &head),
commits,
base_url: &OPT.base_url,
head: &head,
hoc,
hoc_pretty: &hoc_pretty,
path: &service_path,
url: &url,
branch,
};
templates::overview(
&mut buf,
VERSION_INFO,
REPO_COUNT.load(Ordering::Relaxed),
repo_info,
)?;
Ok(HttpResponse::Ok().content_type("text/html").body(buf))
}
};
handle_hoc_request::<T, _>(state, data, branch, mapper).await
}
#[get("/")]
async fn index() -> Result<HttpResponse> {
let mut buf = Vec::new();
templates::index(
&mut buf,
VERSION_INFO,
REPO_COUNT.load(Ordering::Relaxed),
&OPT.base_url,
)?;
Ok(HttpResponse::Ok().content_type("text/html").body(buf))
}
#[post("/generate")]
async fn generate(params: web::Form<GeneratorForm<'_>>) -> Result<HttpResponse> {
let repo = format!("{}/{}", params.user, params.repo);
let mut buf = Vec::new();
templates::generate(
&mut buf,
VERSION_INFO,
REPO_COUNT.load(Ordering::Relaxed),
&OPT.base_url,
params.service.url(),
params.service.service(),
&repo,
)?;
Ok(HttpResponse::Ok().content_type("text/html").body(buf))
}
fn p404() -> Result<HttpResponse> {
let mut buf = Vec::new();
templates::p404(&mut buf, VERSION_INFO, REPO_COUNT.load(Ordering::Relaxed))?;
Ok(HttpResponse::NotFound().content_type("text/html").body(buf))
}
async fn async_p404() -> Result<HttpResponse> {
p404()
}
fn css() -> HttpResponse {
HttpResponse::Ok().content_type("text/css").body(CSS)
}
fn favicon32() -> HttpResponse {
HttpResponse::Ok().content_type("image/png").body(FAVICON)
}
async fn start_server() -> std::io::Result<()> {
let interface = format!("{}:{}", OPT.host, OPT.port);
let state = Arc::new(State {
repos: OPT.repodir.display().to_string(),
cache: OPT.cachedir.display().to_string(),
});
HttpServer::new(move || {
App::new()
.data(state.clone())
.wrap(tracing_actix_web::TracingLogger)
.wrap(middleware::NormalizePath::new(TrailingSlash::Trim))
.service(index)
.service(web::resource("/tacit-css.min.css").route(web::get().to(css)))
.service(web::resource("/favicon.ico").route(web::get().to(favicon32)))
.service(generate)
.service(web::resource("/github/{user}/{repo}").to(calculate_hoc::<GitHub>))
.service(web::resource("/gitlab/{user}/{repo}").to(calculate_hoc::<Gitlab>))
.service(web::resource("/bitbucket/{user}/{repo}").to(calculate_hoc::<Bitbucket>))
.service(
web::resource("/github/{user}/{repo}/delete")
.route(web::post().to(delete_repo_and_cache::<GitHub>)),
)
.service(
web::resource("/gitlab/{user}/{repo}/delete")
.route(web::post().to(delete_repo_and_cache::<Gitlab>)),
)
.service(
web::resource("/bitbucket/{user}/{repo}/delete")
.route(web::post().to(delete_repo_and_cache::<Bitbucket>)),
)
.service(web::resource("/github/{user}/{repo}/json").to(json_hoc::<GitHub>))
.service(web::resource("/gitlab/{user}/{repo}/json").to(json_hoc::<Gitlab>))
.service(web::resource("/bitbucket/{user}/{repo}/json").to(json_hoc::<Bitbucket>))
.service(web::resource("/view/github/{user}/{repo}").to(overview::<GitHub>))
.service(web::resource("/view/gitlab/{user}/{repo}").to(overview::<Gitlab>))
.service(web::resource("/view/bitbucket/{user}/{repo}").to(overview::<Bitbucket>))
.service(web::resource("/github/{user}/{repo}/view").to(overview::<GitHub>))
.service(web::resource("/gitlab/{user}/{repo}/view").to(overview::<Gitlab>))
.service(web::resource("/bitbucket/{user}/{repo}/view").to(overview::<Bitbucket>))
.default_service(web::resource("").route(web::get().to(async_p404)))
})
.workers(OPT.workers)
.bind(interface)?
.run()
.await
telemetry::init_subscriber(telemetry::get_subscriber("hoc", "info"))
}
#[actix_rt::main]
async fn main() -> std::io::Result<()> {
config::init();
let span = info_span!("hoc", version = env!("CARGO_PKG_VERSION"));
let _ = span.enter();
start_server().instrument(span).await
init();
// TODO: error handling
let settings = Settings::load().expect("Cannot load config");
let address = format!("{}:{}", settings.host, settings.port);
// TODO: error handling
let listener = TcpListener::bind(address)?;
hoc::run(listener, settings)
.await
.expect("Server error")
.await
}

View File

@ -1,6 +1,3 @@
use crate::{config::Settings, count::count_repositories};
use std::sync::atomic::AtomicUsize;
pub struct VersionInfo<'a> {
pub commit: &'a str,
pub version: &'a str,
@ -15,7 +12,4 @@ pub(crate) const FAVICON: &[u8] = include_bytes!("../static/favicon32.png");
lazy_static! {
pub(crate) static ref CLIENT: reqwest::Client = reqwest::Client::new();
pub(crate) static ref OPT: Settings = Settings::new().unwrap();
pub(crate) static ref REPO_COUNT: AtomicUsize =
AtomicUsize::new(count_repositories(&OPT.repodir).unwrap());
}

21
src/telemetry.rs Normal file
View File

@ -0,0 +1,21 @@
use tracing::{subscriber::set_global_default, Subscriber};
use tracing_bunyan_formatter::{BunyanFormattingLayer, JsonStorageLayer};
use tracing_log::LogTracer;
use tracing_subscriber::{layer::SubscriberExt, EnvFilter, Registry};
pub fn get_subscriber(name: &str, env_filter: &str) -> impl Subscriber + Send + Sync {
let env_filter =
EnvFilter::try_from_default_env().unwrap_or_else(|_| EnvFilter::new(env_filter));
let formatting_layer = BunyanFormattingLayer::new(name.to_string(), std::io::stdout);
Registry::default()
.with(env_filter)
.with(JsonStorageLayer)
.with(formatting_layer)
}
pub fn init_subscriber(subscriber: impl Subscriber + Send + Sync) {
LogTracer::init().expect("Failed to set logger");
set_global_default(subscriber).expect("Failed to set tracing subscriber");
}

View File

@ -1,68 +0,0 @@
use crate::{
calculate_hoc, index, json_hoc,
service::{Bitbucket, GitHub, Gitlab, Service},
State,
};
use actix_web::{http, test, web, App};
use tempfile::tempdir;
macro_rules! test_app {
($path: expr) => {
test::init_service(App::new().service($path)).await
};
($state: expr, $path: expr) => {
test::init_service(App::new().data($state).service($path)).await
};
}
macro_rules! test_service {
($name: ident, $path: tt, $what: ident) => {
async fn $name<T: 'static + Service>(req_path: &str) {
let repo_dir = dbg!(tempdir().unwrap());
let cache_dir = dbg!(tempdir().unwrap());
let repos = format!("{}/", repo_dir.path().display());
let cache = format!("{}/", cache_dir.path().display());
let state = dbg!(State { repos, cache });
let mut app = test_app!(state, web::resource($path).to($what::<T>));
let req = dbg!(test::TestRequest::with_uri(req_path).to_request());
let resp = dbg!(test::call_service(&mut app, req).await);
assert_eq!(resp.status(), http::StatusCode::OK);
}
};
}
#[actix_rt::test]
async fn test_index() {
std::env::set_var("HOC_BASE_URL", "http://0.0.0.0:8080");
let mut app = test_app!(index);
let req = dbg!(test::TestRequest::with_uri("/").to_request());
let resp = dbg!(test::call_service(&mut app, req).await);
assert_eq!(resp.status(), http::StatusCode::OK);
}
// TODO: fix this test
// #[actix_rt::test]
async fn test_json() {
test_service!(test_json_service, "/service/{user}/{repo}/json", json_hoc);
test_json_service::<Gitlab>("/service/vbrandl/hoc/json").await;
test_json_service::<GitHub>("/service/vbrandl/hoc/json").await;
test_json_service::<Bitbucket>("/service/vbrandl/hoc/json").await;
}
// TODO: fix this test
// #[actix_rt::test]
async fn test_badge() {
test_service!(test_badge_service, "/service/{user}/{repo}", calculate_hoc);
test_badge_service::<Gitlab>("/service/vbrandl/hoc").await;
test_badge_service::<GitHub>("/service/vbrandl/hoc").await;
test_badge_service::<Bitbucket>("/service/vbrandl/hoc").await;
}

View File

@ -19,7 +19,7 @@ To include the badge in your readme, use the following markdown:
</p>
<pre>
[![Hits-of-Code](@repo_info.base_url/@repo_info.path?branch=@repo_info.branch)](@repo_info.base_url/@repo_info.path?branch=@repo_info.branch/view?branch=@repo_info.branch)
[![Hits-of-Code](@repo_info.base_url/@repo_info.path?branch=@repo_info.branch)](@repo_info.base_url/@repo_info.path/view?branch=@repo_info.branch)
</pre>

18
tests/badge.rs Normal file
View File

@ -0,0 +1,18 @@
mod util;
use actix_web::client;
#[actix_rt::test]
async fn badge_succeeds() {
let test_app = util::spawn_app().await;
let client = client::Client::default();
let response = client
.get(&format!("{}/github/vbrandl/hoc", test_app.address))
.send()
.await
.expect("Failed to execute request");
assert!(response.status().is_success());
}

18
tests/health_check.rs Normal file
View File

@ -0,0 +1,18 @@
mod util;
use actix_web::client;
#[actix_rt::test]
async fn health_check_works() {
let test_app = util::spawn_app().await;
let client = client::Client::default();
let response = client
.get(&format!("{}/health_check", test_app.address))
.send()
.await
.expect("Failed to execute request");
assert!(response.status().is_success());
}

18
tests/index.rs Normal file
View File

@ -0,0 +1,18 @@
mod util;
use actix_web::client;
#[actix_rt::test]
async fn index_returns_success() {
let test_app = util::spawn_app().await;
let client = client::Client::default();
let response = client
.get(&format!("{}/", test_app.address))
.send()
.await
.expect("Failed to execute request");
assert!(response.status().is_success());
}

18
tests/json.rs Normal file
View File

@ -0,0 +1,18 @@
mod util;
use actix_web::client;
#[actix_rt::test]
async fn json_returns_success() {
let test_app = util::spawn_app().await;
let client = client::Client::default();
let response = client
.get(&format!("{}/github/vbrandl/hoc/json", test_app.address))
.send()
.await
.expect("Failed to execute request");
assert!(response.status().is_success());
}

50
tests/util/mod.rs Normal file
View File

@ -0,0 +1,50 @@
use hoc::{config::Settings, telemetry};
use std::net::TcpListener;
use tempfile::{tempdir, TempDir};
lazy_static::lazy_static! {
static ref TRACING: () = {
let filter = if std::env::var("TEST_LOG").is_ok() { "debug" } else { "" };
let subscriber = telemetry::get_subscriber("test", filter);
telemetry::init_subscriber(subscriber);
};
}
pub struct TestApp {
pub address: String,
repo_dir: TempDir,
cache_dir: TempDir,
}
pub async fn spawn_app() -> TestApp {
lazy_static::initialize(&TRACING);
let listener = TcpListener::bind("127.0.0.1:0").expect("Failed to bind random port");
let port = listener.local_addr().unwrap().port();
let address = format!("http://127.0.0.1:{}", port);
let repo_dir = tempdir().expect("Cannot create repo_dir");
let cache_dir = tempdir().expect("Cannot create cache_dir");
let mut settings = Settings::load().expect("Failed to read configuration.");
settings.repodir = repo_dir.path().to_path_buf();
settings.cachedir = cache_dir.path().to_path_buf();
// configuration.database.database_name = Uuid::new_v4().to_string();
// let connection_pool = configure_database(&configuration.database).await;
let server = hoc::run(listener, settings)
.await
.expect("Failed to bind address");
let _ = tokio::spawn(server);
TestApp {
address,
repo_dir,
cache_dir,
}
}