remove num_cpus dependency (#488)

2024-11-23 15:31:07 +01:00 · 2023-08-24 00:56:31 +01:00 · 2023-08-24 00:56:31 +01:00 · 6ce8307060
commit 6ce8307060
parent 9cb8a1fadc
6 changed files with 115 additions and 49 deletions
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@ -37,6 +37,10 @@ jobs:
        run: sudo ifconfig lo0 alias 127.0.0.3

      - uses: actions/checkout@v3
+      
+      - name: Free Disk Space
+        if: matrix.target.os == 'ubuntu-latest'
+        run: ./scripts/free-disk-space.sh

      - name: Install OpenSSL
        if: matrix.target.os == 'windows-latest'
--- a/actix-server/CHANGES.md
+++ b/actix-server/CHANGES.md
@ -1,22 +1,22 @@
 # Changes

-## Unreleased - 2023-xx-xx
+## Unreleased

 - Add support for MultiPath TCP (MPTCP) with `MpTcp` enum and `ServerBuilder::mptcp()` method.
 - Minimum supported Rust version (MSRV) is now 1.65.

-## 2.2.0 - 2022-12-21
+## 2.2.0

 - Minimum supported Rust version (MSRV) is now 1.59.
 - Update `tokio-uring` dependency to `0.4`. [#473]

 [#473]: https://github.com/actix/actix-net/pull/473

-## 2.1.1 - 2022-03-09
+## 2.1.1

 - No significant changes since `2.1.0`.

-## 2.1.0 - 2022-03-08
+## 2.1.0

 - Update `tokio-uring` dependency to `0.3`. [#448]
 - Logs emitted now use the `tracing` crate with `log` compatibility. [#448]
@ -25,27 +25,27 @@
 [#443]: https://github.com/actix/actix-net/pull/443
 [#448]: https://github.com/actix/actix-net/pull/448

-## 2.0.0 - 2022-01-19
+## 2.0.0

 - No significant changes since `2.0.0-rc.4`.

-## 2.0.0-rc.4 - 2022-01-12
+## 2.0.0-rc.4

 - Update `tokio-uring` dependency to `0.2`. [#436]

 [#436]: https://github.com/actix/actix-net/pull/436

-## 2.0.0-rc.3 - 2021-12-31
+## 2.0.0-rc.3

 - No significant changes since `2.0.0-rc.2`.

-## 2.0.0-rc.2 - 2021-12-27
+## 2.0.0-rc.2

 - Simplify `TestServer`. [#431]

 [#431]: https://github.com/actix/actix-net/pull/431

-## 2.0.0-rc.1 - 2021-12-05
+## 2.0.0-rc.1

 - Hide implementation details of `Server`. [#424]
 - `Server` now runs only after awaiting it. [#425]
@ -53,19 +53,19 @@
 [#424]: https://github.com/actix/actix-net/pull/424
 [#425]: https://github.com/actix/actix-net/pull/425

-## 2.0.0-beta.9 - 2021-11-15
+## 2.0.0-beta.9

 - Restore `Arbiter` support lost in `beta.8`. [#417]

 [#417]: https://github.com/actix/actix-net/pull/417

-## 2.0.0-beta.8 - 2021-11-05 _(YANKED)_
+## 2.0.0-beta.8

 - Fix non-unix signal handler. [#410]

 [#410]: https://github.com/actix/actix-net/pull/410

-## 2.0.0-beta.7 - 2021-11-05 _(YANKED)_
+## 2.0.0-beta.7

 - Server can be started in regular Tokio runtime. [#408]
 - Expose new `Server` type whose `Future` impl resolves when server stops. [#408]
@ -78,7 +78,7 @@
 [#407]: https://github.com/actix/actix-net/pull/407
 [#408]: https://github.com/actix/actix-net/pull/408

-## 2.0.0-beta.6 - 2021-10-11
+## 2.0.0-beta.6

 - Add experimental (semver-exempt) `io-uring` feature for enabling async file I/O on linux. [#374]
 - Server no long listens to `SIGHUP` signal. Previously, the received was not used but did block subsequent exit signals from working. [#389]
@ -89,19 +89,19 @@
 [#349]: https://github.com/actix/actix-net/pull/349
 [#389]: https://github.com/actix/actix-net/pull/389

-## 2.0.0-beta.5 - 2021-04-20
+## 2.0.0-beta.5

 - Server shutdown notifies all workers to exit regardless if shutdown is graceful. This causes all workers to shutdown immediately in force shutdown case. [#333]

 [#333]: https://github.com/actix/actix-net/pull/333

-## 2.0.0-beta.4 - 2021-04-01
+## 2.0.0-beta.4

 - Prevent panic when `shutdown_timeout` is very large. [f9262db]

 [f9262db]: https://github.com/actix/actix-net/commit/f9262db

-## 2.0.0-beta.3 - 2021-02-06
+## 2.0.0-beta.3

 - Hidden `ServerBuilder::start` method has been removed. Use `ServerBuilder::run`. [#246]
 - Add retry for EINTR signal (`io::Interrupted`) in `Accept`'s poll loop. [#264]
@ -113,13 +113,13 @@
 [#265]: https://github.com/actix/actix-net/pull/265
 [#273]: https://github.com/actix/actix-net/pull/273

-## 2.0.0-beta.2 - 2021-01-03
+## 2.0.0-beta.2

 - Merge `actix-testing` to `actix-server` as `test_server` mod. [#242]

 [#242]: https://github.com/actix/actix-net/pull/242

-## 2.0.0-beta.1 - 2020-12-28
+## 2.0.0-beta.1

 - Added explicit info log message on accept queue pause. [#215]
 - Prevent double registration of sockets when back-pressure is resolved. [#223]
@ -134,127 +134,127 @@
 [#223]: https://github.com/actix/actix-net/pull/223
 [#239]: https://github.com/actix/actix-net/pull/239

-## 1.0.4 - 2020-09-12
+## 1.0.4

 - Update actix-codec to 0.3.0.
 - Workers must be greater than 0. [#167]

 [#167]: https://github.com/actix/actix-net/pull/167

-## 1.0.3 - 2020-05-19
+## 1.0.3

 - Replace deprecated `net2` crate with `socket2` [#140]

 [#140]: https://github.com/actix/actix-net/pull/140

-## 1.0.2 - 2020-02-26
+## 1.0.2

 - Avoid error by calling `reregister()` on Windows [#103]

 [#103]: https://github.com/actix/actix-net/pull/103

-## 1.0.1 - 2019-12-29
+## 1.0.1

 - Rename `.start()` method to `.run()`

-## 1.0.0 - 2019-12-11
+## 1.0.0

 - Use actix-net releases

-## 1.0.0-alpha.4 - 2019-12-08
+## 1.0.0-alpha.4

 - Use actix-service 1.0.0-alpha.4

-## 1.0.0-alpha.3 - 2019-12-07
+## 1.0.0-alpha.3

 - Migrate to tokio 0.2
 - Fix compilation on non-unix platforms
 - Better handling server configuration

-## 1.0.0-alpha.2 - 2019-12-02
+## 1.0.0-alpha.2

 - Simplify server service (remove actix-server-config)
 - Allow to wait on `Server` until server stops

-## 0.8.0-alpha.1 - 2019-11-22
+## 0.8.0-alpha.1

 - Migrate to `std::future`

-## 0.7.0 - 2019-10-04
+## 0.7.0

 - Update `rustls` to 0.16
 - Minimum required Rust version upped to 1.37.0

-## 0.6.1 - 2019-09-25
+## 0.6.1

 - Add UDS listening support to `ServerBuilder`

-## 0.6.0 - 2019-07-18
+## 0.6.0

 - Support Unix domain sockets #3

-## 0.5.1 - 2019-05-18
+## 0.5.1

 - ServerBuilder::shutdown_timeout() accepts u64

-## 0.5.0 - 2019-05-12
+## 0.5.0

 - Add `Debug` impl for `SslError`
 - Derive debug for `Server` and `ServerCommand`
 - Upgrade to actix-service 0.4

-## 0.4.3 - 2019-04-16
+## 0.4.3

 - Re-export `IoStream` trait
 - Depend on `ssl` and `rust-tls` features from actix-server-config

-## 0.4.2 - 2019-03-30
+## 0.4.2

 - Fix SIGINT force shutdown

-## 0.4.1 - 2019-03-14
+## 0.4.1

 - `SystemRuntime::on_start()` - allow to run future before server service initialization

-## 0.4.0 - 2019-03-12
+## 0.4.0

 - Use `ServerConfig` for service factory
 - Wrap tcp socket to `Io` type
 - Upgrade actix-service

-## 0.3.1 - 2019-03-04
+## 0.3.1

 - Add `ServerBuilder::maxconnrate` sets the maximum per-worker number of concurrent connections
 - Add helper ssl error `SslError`
 - Rename `StreamServiceFactory` to `ServiceFactory`
 - Deprecate `StreamServiceFactory`

-## 0.3.0 - 2019-03-02
+## 0.3.0

 - Use new `NewService` trait

-## 0.2.1 - 2019-02-09
+## 0.2.1

 - Drop service response

-## 0.2.0 - 2019-02-01
+## 0.2.0

 - Migrate to actix-service 0.2
 - Updated rustls dependency

-## 0.1.3 - 2018-12-21
+## 0.1.3

 - Fix max concurrent connections handling

-## 0.1.2 - 2018-12-12
+## 0.1.2

 - rename ServiceConfig::rt() to ServiceConfig::apply()
 - Fix back-pressure for concurrent ssl handshakes

-## 0.1.1 - 2018-12-11
+## 0.1.1

 - Fix signal handling on windows

-## 0.1.0 - 2018-12-09
+## 0.1.0

 - Move server to separate crate
--- a/actix-server/Cargo.toml
+++ b/actix-server/Cargo.toml
@ -27,7 +27,6 @@ actix-utils = "3"
 futures-core = { version = "0.3.17", default-features = false, features = ["alloc"] }
 futures-util = { version = "0.3.17", default-features = false, features = ["alloc"] }
 mio = { version = "0.8", features = ["os-poll", "net"] }
-num_cpus = "1.13"
 socket2 = "0.5"
 tokio = { version = "1.23.1", features = ["sync"] }
 tracing = { version = "0.1.30", default-features = false, features = ["log"] }
--- a/actix-server/src/builder.rs
+++ b/actix-server/src/builder.rs
@ -1,4 +1,4 @@
-use std::{io, time::Duration};
+use std::{io, num::NonZeroUsize, time::Duration};

 use actix_rt::net::TcpStream;
 use tokio::sync::mpsc::{unbounded_channel, UnboundedReceiver, UnboundedSender};
@ -55,7 +55,7 @@ impl ServerBuilder {
        let (cmd_tx, cmd_rx) = unbounded_channel();

        ServerBuilder {
-            threads: num_cpus::get_physical(),
+            threads: std::thread::available_parallelism().map_or(2, NonZeroUsize::get),
            token: 0,
            factories: Vec::new(),
            sockets: Vec::new(),
@ -76,6 +76,12 @@ impl ServerBuilder {
    /// The default worker count is the number of physical CPU cores available. If your benchmark
    /// testing indicates that simultaneous multi-threading is beneficial to your app, you can use
    /// the [`num_cpus`] crate to acquire the _logical_ core count instead.
+    ///
+    /// # Panics
+    ///
+    /// Panics if `num` is 0.
+    ///
+    /// [`num_cpus`]: https://docs.rs/num_cpus
    pub fn workers(mut self, num: usize) -> Self {
        assert_ne!(num, 0, "workers must be greater than 0");
        self.threads = num;
--- a/actix-server/src/worker.rs
+++ b/actix-server/src/worker.rs
@ -1,6 +1,7 @@
 use std::{
    future::Future,
    io, mem,
+    num::NonZeroUsize,
    pin::Pin,
    rc::Rc,
    sync::{
@ -249,8 +250,11 @@ pub(crate) struct ServerWorkerConfig {

 impl Default for ServerWorkerConfig {
    fn default() -> Self {
-        // 512 is the default max blocking thread count of tokio runtime.
-        let max_blocking_threads = std::cmp::max(512 / num_cpus::get_physical(), 1);
+        let parallelism = std::thread::available_parallelism().map_or(2, NonZeroUsize::get);
+
+        // 512 is the default max blocking thread count of a Tokio runtime.
+        let max_blocking_threads = std::cmp::max(512 / parallelism, 1);
+
        Self {
            shutdown_timeout: Duration::from_secs(30),
            max_blocking_threads,
--- a/scripts/free-disk-space.sh
+++ b/scripts/free-disk-space.sh
@ -0,0 +1,53 @@
+#!/usr/bin/env bash
+
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# The Azure provided machines typically have the following disk allocation:
+# Total space: 85GB
+# Allocated: 67 GB
+# Free: 17 GB
+# This script frees up 28 GB of disk space by deleting unneeded packages and 
+# large directories.
+# The Flink end to end tests download and generate more than 17 GB of files,
+# causing unpredictable behavior and build failures.
+
+echo "=============================================================================="
+echo "Freeing up disk space on CI system"
+echo "=============================================================================="
+
+echo "Listing 100 largest packages"
+dpkg-query -Wf '${Installed-Size}\t${Package}\n' | sort -n | tail -n 100
+df -h
+
+echo "Removing large packages"
+sudo apt-get remove -y '^dotnet-.*'
+sudo apt-get remove -y 'php.*'
+sudo apt-get remove -y '^mongodb-.*'
+sudo apt-get remove -y '^mysql-.*'
+sudo apt-get remove -y azure-cli google-cloud-sdk hhvm google-chrome-stable firefox powershell mono-devel libgl1-mesa-dri
+sudo apt-get autoremove -y
+sudo apt-get clean
+df -h
+
+echo "Removing large directories"
+sudo rm -rf /usr/share/dotnet/
+sudo rm -rf /usr/local/graalvm/
+sudo rm -rf /usr/local/.ghcup/
+sudo rm -rf /usr/local/share/powershell
+sudo rm -rf /usr/local/share/chromium
+sudo rm -rf /usr/local/lib/android
+sudo rm -rf /usr/local/lib/node_modules
+df -h