* Apache 2 (nginx and IIS also supported). A fast internet connection is required (possibly gigabit), and the web server must accept large POST requests (up to 20MB)
* PHP 5.4 or newer, a 64-bit version is strongly recommended
* OpenSSL and its PHP module (this is usually installed automatically by most distros)
* If you want to store test results (telemetry), one of the following:
- MySQL/MariaDB and the mysqli PHP module
- PostgreSQL and its PHP PDO module
- SQLite 3 and its PHP PDO module
* If you want to enable results sharing:
- FreeType 2 and its PHP module (this is usually installed automatically by most distros)
Let's install the speedtest.
Put all files on your web server via FTP or by copying them directly. You can install it in the root, or in a subdirectory.
__Important:__ The speedtest needs write permissions in the installation folder!
#### ipinfo.io
The speedtest uses [ipinfo.io](https://ipinfo.io) to detect ISP and distance from server. This is completely optional and can be disabled if you want (see Speedtest settings), but it is enabled by default, and if you expect more than ~500 tests per day, you will need to sign up to [ipinfo.io](https://ipinfo.io) and edit `backend/getIP_ipInfo_apikey.php` to set your access token.
IpInfo.io has kindly offered free access to their APIs for users of this project; if you're interested, contact me at [info@fdossena.com](mailto:info@fdossena.com) and provide a description of what you intend to do with the project, and you'll get the API key.
#### Telemetry and results sharing
The test supports storing test results and can generate shareable images that users can embed in forum signatures and such.
To use this function, you will need a database. The test supports MySQL, PostgreSQL and SQLite as backends.
##### Creating the database
This step is only required for MySQL and PostgreSQL. If you want to use SQLite, skip to the next step.
Log into your database using phpMyAdmin or a similar software and create a new database. Inside the `results` folder you will find `telemetry_mysql.sql` and `telemetry_postgresql.sql`, which are templates for MySQL and PostgreSQL respectively. Import the one you need, and you will see a `speedtest_users` table in the database. You can delete the templates afterwards.
##### Configuring telemetry
Open `results/telemetry_settings.php` in a text editor. Set `$db_type` to either `mysql`,`postgresql` or `sqlite`.
If you chose to use SQLite, you might want to change `$Sqlite_db_file` to another path where you want the database to be stored. Just make sure that the file cannot be downloaded by users. Sqlite doesn't require any additional configuration, you can skip the rest of this section.
If you chose to use MySQL, you must set your database credentials:
This feature generates an image that can be share by the user containing the download, upload, ping, jitter and ISP (if enabled).
By default, the telemetry generates a progressive ID for each test. Even if no sensitive information is leaked, you might not want users to be able to guess other test IDs. To avoid this, you can turn on ID obfuscation, which turns IDs into a reversible hash, much like YouTube video IDs.
To enable ID obfuscation, edit `results/telemetry_settings.php` and set `$enable_id_obfuscation` to `true`. From now on, all test IDs will be obfuscated using a unique salt. The IDs in the database are still progressive, but users will only know their obfuscated versions and won't be able to easily guess other IDs.
__Important:__ ID obfuscation currently only works on 64-bit PHP!
While you're editing `results/telemetry_settings.php`, you might want to set `$redact_ip_addresses` to `true`, this way, all IP addresses will be removed from the telemetry for better privacy. This is disabled by default.
The best starting point for most people is `example-singleServer-gauges.html`. If you want to use telemetry and results sharing, use `example-singleServer-full.html` instead.
Details about the examples and how to make custom UIs will be discussed later. If you don't want to make a custom UI, feel free to modify the example and replace "LibreSpeed Example" with the name of your test.
Telemetry contains personal information (according to GDPR defintion), therefore it is important to treat this data respectfully of national and international laws, especially if you plan to offer the service in the European Union.
`example-singleServer-full.html` and `example-multipleServers-full.html` both contain a privacy policy for the service: you MUST read it, change it if necessary, and add your email address for data deletion requests. __Failure to comply with GDPR regulations can get you in serious trouble.__
The speedtest can automatically choose between multiple test points and use the one with the lowest ping in a list.
Note that this is an advanced use case and it is recommended that you already know how to use the speedtest with a single server.
We must distinguish 2 types of servers:
* __Frontend server__: hosts the UI, the JS files, and optionally telemetry and results sharing stuff. You only need 1 of these, and this is the server that your clients will first connect to.
* __Test backends__: the servers used to actually perform the test. There can be 1+ of these, and they only host the backend files.
#### Frontend server
This is the server that your users will first connect to. It hosts the UI, the JS files, and optionally telemetry and results sharing stuff.
Requirements:
* Apache 2 (nginx and IIS also supported). A fast connection is not mandatory, but is still recommended
* PHP 5.4 or newer
* If you want to store test results (telemetry), one of the following:
- MySQL/MariaDB and the mysqli PHP module
- PostgreSQL and its PHP PDO module
- SQLite 3 and its PHP PDO module
* If you want to enable results sharing:
- FreeType 2 and its PHP module (this is usually installed automatically by most distros)
To install the speedtest frontend, copy the following files to your web server:
*`speedtest.js`
*`speedtest_worker.js`
* Optionally, the `results` folder
* One of the `multipleServers` examples (the best starting points are `example-multipleServers-pretty.html` if you don't want to use telemetry and results sharing, `example-multipleServers-full.html` if you want to use them). Rename the example you choose to `index.html`
__Important:__ The speedtest needs write permissions in the installation folder!
##### Server list
Edit `index.html`, you will see a list of servers:
Replace the demo servers with your test points. Each server in the list is an object containing:
*`name`: user friendly name for this test point
*`server`: URL to the server. If your server only supports HTTP or HTTPS, put http:// or https:// at the beginning, respectively; if it supports both, put // at the beginning and it will be replaced automatically
*`dlURL`: path to the download test on this server (garbage.php or replacement)
*`ulURL`: path to the upload test on this server (empty.php or replacement)
*`pingURL`: path to the ping test on this server (empty.php or replacement)
*`getIpURL`: path to getIP on this server (getIP.php or replacement)
__Important__: You can't mix HTTP with HTTPS; if the frontend uses HTTP, you won't be able to connect to HTTPS backends, and viceversa.
__Important__: For HTTPS, all your servers must have valid certificates or the browser will refuse to connect
__Important__: Don't use my demo servers, they're slow!
##### Telemetry and results sharing
Telemetry is stored on the frontend server. The setup procedure is the same as the single server version.
#### Test backends
These are the servers that will actually be used to perform the test.
Requirements:
* Apache 2 (nginx and IIS also supported). A fast internet connection is required (possibly gigabit), and the web server must accept large POST requests (up to 20MB)
* PHP 5.4 or newer
* OpenSSL and its PHP module (this is usually installed automatically by most distros)
To install a backend, simply copy all the files in the `backend` folder to your backend server.
__Important:__ The speedtest needs write permissions in the installation folder!
#### ipinfo.io
The speedtest uses [ipinfo.io](https://ipinfo.io) to detect ISP and distance from server. This is completely optional and can be disabled if you want (see Speedtest settings), but it is enabled by default, and if you expect more than ~500 tests per day, you will need to sign up to [ipinfo.io](https://ipinfo.io) and edit `getIP_ipInfo_apikey.php` to set your access token.
IpInfo.io has kindly offered free access to their APIs for users of this project; if you're interested, contact me at [info@fdossena.com](mailto:info@fdossena.com) and provide a description of what you intend to do with the project, and you'll get the API key.
## Making a custom front-end
This section explains how to use speedtest.js in your webpages.
The best way to learn is by looking at the provided examples.
__Single server:__
*`example-singleServer-basic.html`: The most basic configuration possible. Runs the test with the default settings when the page is loaded and displays the results with no fancy graphics.
*`example-singleServer-pretty.html`: A more sophisticated example with a nicer layout and a start/stop button. __This is the best starting point for most users__
*`example-singleServer-progressBar.html`: Same as `example-singleServer-pretty.html` but adds a progress indicator
*`example-singleServer-customSettings.html`: Same as `example-singleServer-pretty.html` but configures the test so that it only performs download and upload tests, and with a fixed length instead of automatic
*`example-singleServer-gauges.html`: The most sophisticated example, with the same functionality as `example-singleServer-pretty.html` but adds gauges. This is also a good starting point, but the gauges may slow down underpowered devices
*`example-singleServer-chart.html`: Shows how to use the test with the Chart.js library
*`example-singleServer-full.html`: The most complete example. Based on `example-singleServer-gauges.html`, also enables telemetry and results sharing
__Multiple servers:__
*`example-multipleServers-pretty.html`: Same as `example-singleServer-pretty.html` but with multiple test points. Server selection is fully automatic
*`example-multipleServers-full.html`: Same as `example-singleServer-full.html` but with multiple test points. Server selection is automatic but the server can be changed afterwards by the user
### Initialization
To use the speedtest in your page, first you need to load it:
The `onupdate` event handler will be called periodically by the test with data coming from the speedtest worker thread. The `data` argument is an object containing the following:
* __testId__: when telemetry is active, this is the ID of the test in the database. This is null until the test is finished, or if telemetry encounters an error. This ID is used for results sharing
The `onend` event handler will be called at the end of the test (`onupdate` will be called first), with a boolean telling you if the test was aborted (either manually or because of an error) or if it ended normally.
Before starting the test, you can change some of the settings from their default values. You might want to do this to better adapt the speedtest to a specific scenario, such as a satellite connection. To change a setting, use
* __time_auto__: Automatically determine the duration of the download and upload tests, making them faster on faster connections, to avoid wasting data.
* __telemetry_level__: The type of telemetry to use. See the telemetry section for more info about this
* Default: `none`
*`basic`: send results only
*`full`: send results and timing information, even for aborted tests
*`debug`: same as full but also sends debug information. Not recommended.
* __test_order__: the order in which tests will be performed. You can use this to change the order of the test, or to only enable specific tests. Each character represents an operation:
* __getIp_ispInfo__: if true, the server will try to get ISP info and pass it along with the IP address. This will add `isp=true` to the request to `url_getIp`. getIP.php accomplishes this using ipinfo.io
* __getIp_ispInfo_distance__: if true, the server will try to get an estimate of the distance from the client to the speedtest server. This will add a `distance` argument to the request to `url_getIp`. `__getIp_ispInfo__` must be enabled in order for this to work. getIP.php accomplishes this using ipinfo.io
__Advanced parameters:__ (Seriously, don't change these unless you know what you're doing)
* __telemetry_extra__: Extra data that you want to be passed to the telemetry. This is a string field, if you want to pass an object, make sure you use ``JSON.stringify``. This string will be added to the database entry for this test.
* __enable_quirks__: enables browser-specific optimizations. These optimizations override some of the default settings. They do not override settings that are explicitly set.
* __time_dlGraceTime__: How long to wait (in seconds) before actually measuring the download speed. This is a good idea because we want to wait for the TCP window to be at its maximum (or close to it)
* Default: `1.5`
* Recommended: `>=0`
* __time_ulGraceTime__: How long to wait (in seconds) before actually measuring the upload speed. This is a good idea because we want to wait for the buffers to be full (avoids the peak at the beginning of the test)
* __overheadCompensationFactor__: compensation for HTTP and network overhead. Default value assumes typical MTUs used over the Internet. You might want to change this if you're using this in your internal network with different MTUs, or if you're using IPv6 instead of IPv4.
* Default: `1.06` probably a decent estimate for all overhead. This was measured empirically by comparing the measured speed and the speed reported by my the network adapter.
*`1048576/925000`: old default value. This is probably too high.
*`1.0513`: HTTP+TCP+IPv6+ETH, over the Internet (empirically tested, not calculated)
If you want to use more than one test server, this is the time to add all your test points and select the best one. Skip this part if you don't want to use this feature.
The best way to do this is to declare an array with all your servers, and give it to the speedtest:
*`server`: URL to the server. If your server only supports HTTP or HTTPS, put `http://` or `https://` at the beginning, respectively; if it supports both, put `//` at the beginning and it will be replaced automatically
*`dlURL`: path to the download test on this server (garbage.php or replacement)
*`ulURL`: path to the upload test on this server (empty.php or replacement)
*`pingURL`: path to the ping test on this server (empty.php or replacement)
*`getIpURL`: path to getIP on this server (getIP.php or replacement)
The `selectServer` function is asynchronous in order to avoid freeing the UI, and it will run a callback function when it is done choosing the server with the lowest ping.
The `server` argument is the selected server, and you can display it in the UI if you want. __You cannot start the test until the selection is done!__
During the test, your `onupdate` event handler will be called periodically with data that you can use to update your UI. Your `onend` handler will be called at the end of the test.
This is the main interface between your webpage and the speedtest.
It hides the speedtest web worker to the page, and provides many convenient functions to control the test.
You can think of this as a finite state machine. These are the states (use getState() to see them):
* __0__: here you can change the speedtest settings (such as test duration) with the `setParameter("parameter",value)` function. From here you can either start the test using `start()` (goes to state 3) or you can add multiple test points using `addTestPoint(server)` or `addTestPoints(serverList)` (goes to state 1). Additionally, this is the perfect moment to set up callbacks for the `onupdate(data)` and `onend(aborted)` events.
* __1__: here you can add test points. You only need to do this if you want to use multiple test points.
While in state 1, you can only add test points, you cannot change the test settings. When you're done, use selectServer(callback) to select the test point with the lowest ping. This is asynchronous, when it's done, it will call your callback function and move to state 2. Calling setSelectedServer(server) will manually select a server and move to state 2.
* __2__: test point selected, ready to start the test. Use `start()` to begin, this will move to state 3
* __3__: test running. Here, your `onupdate` event calback will be called periodically, with data coming from the worker about speed and progress. A data object will be passed to your `onupdate` function, with the following items:
-`dlStatus`: download speed in mbps
-`ulStatus`: upload speed in mbps
-`pingStatus`: ping in ms
-`jitterStatus`: jitter in ms
-`dlProgress`: progress of the download test as a float 0-1
-`ulProgress`: progress of the upload test as a float 0-1
-`pingProgress`: progress of the ping/jitter test as a float 0-1
-`testState`: state of the test (-1=not started, 0=starting, 1=download test, 2=ping+jitter test, 3=upload test, 4=finished, 5=aborted)
-`clientIp`: IP address of the client performing the test (and optionally ISP and distance)
At the end of the test, the `onend` function will be called, with a boolean specifying whether the test was aborted or if it ended normally.
The test can be aborted at any time with `abort()`.
At the end of the test, it will move to state 4
* __4__: test finished. You can run it again by calling `start()` if you want.
- server: the server to be added as an object. Must contain the following elements:
```
{
name: "User friendly name",
server:"http://yourBackend.com/", URL to your server. You can specify http:// or https://. If your server supports both, just write // without the protocol
dlURL:"garbage.php" path to garbage.php or its replacement on the server
ulURL:"empty.php" path to empty.php or its replacement on the server
pingURL:"empty.php" path to empty.php or its replacement on the server. This is used to ping the server by this selector
getIpURL:"getIP.php" path to getIP.php or its replacement on the server
}
```
Note that this will add `mpot`:`true` to the parameters sent to the speedtest worker.
##### addTestPoints(list)
Same as addTestPoint, but you can pass an array of servers
##### getSelectedServer()
Returns the selected server (multiple points of test)
##### setSelectedServer()
Manually selects one of the test points (multiple points of test)
##### selectServer(result)
Automatically selects a server from the list of added test points. The server with the lowest ping will be chosen. (multiple points of test)
The selector checks multiple servers in parallel (default: 6 streams) to speed things up if the list of servers is long.
The process is asynchronous and the passed `result` callback function will be called when it's done, then the test can be started.
##### start()
Starts the test.
Note (multiple points of test): the selected server will be added to the `telemetry_extra` string. If this string was already set, then `telemetry_extra` will be a JSON string containing both the server and the original string
During the test, the `onupdate(data)` callback function will be called periodically with data from the worker.
At the end of the test, the `onend(aborted)` function will be called with a boolean telling you if the test was aborted or if it ended normally.
##### abort()
Aborts the test while it's running.
### `speedtest_worker.js`
This is where the actual speedtest code is. It receives the settings from the main thread, runs the test, and reports back the results.
The worker accepts 3 commands:
*`start`: starts the test. Optionally, test settings can be passed as a JSON string after the word start and a space
*`status`: returns the current status as a JSON string. The status string contents are the ones described in the Event handlers section in the section about making a custom front-end.
*`abort`: aborts the test
#### Parameters
In addition to the parameters listed in the Test settings section in the section about making a custom front-end, there is one additional setting:
*`mpot`: set this to true to run the test with multiple points of test. This will add `cors=true` to all requests (all responses will contain CORS headers) and enable some extra quirks.
Default: `false`
#### Download test
The download test is performed by transferring large blobs of garbage data using XHR from the server to the client.
The test uses multiple streams. If a stream finishes the download, it is restarted. The amount of downloaded data for each stream is tracked using the XHR Level 2 `onprogress` event.
The test streams are not perfectly synchronized because we don't want them to finish all at the same time if they do.
Every 200ms, a timer updates the `dlStatus` string with the current speed and calculates a "bonus" time by which to shorten the test depending on how high the speed is, (when `time_auto` is set to `true`).
See the code for more implementation details.
#### Upload test
This works similarly to the download test, but in reverse. A large blob of garbage data is generated and it is sent to the server repeatedly using multiple streams.
To keep track of the amount of transferred data, the XHR Level 2 `upload.onprogress` event is used.
This test has a couple of complications:
* Some browsers don't have a working `upload.onprogress` event. For this, we use a small blobs instead of a large one and we keep track of progress using the `onload` event. This is referred to as IE11 Workaround (but the same bug was also found in some versions of Edge and Safari)
* When `mpot` is set to `true`, an empty request must first be sent in order to load the CORS headers before the test can start
This test works by creating a persistent HTTP connection to the server, and then repeatedly downloading an empty file, and measuring how long it takes between the request and the response.
Timing can be measured as a simple timestamp difference or with the Performance API if available.
Jitter is the variance in ping times.
See the code for more implementation details.
### `backend` files
#### `garbage.php`
Uses OpenSSL to generate a stream of incompressible garbage data for the download test.
If accepts a `ckSize` GET parameter, which specifies how much garbage data to generate in megabytes (4-1024).
#### `empty.php`
An empty file used for the upload and ping test. It only sends headers to create the connection.
#### `getIP.php`
Returns client IP, ISP and distance from the server.
GET parameters:
*`isp`: if set, fetches ISP info from ipinfo.io
*`distance`: if set, calculates distance from server. You can specify `km` or `mi` for the format.
If `isp` is set, the output is a JSON string containing:
*`processedString`: string that can be displayed to the user
*`rawIspInfo`: info about the client as a JSON string, straight from ipinfo.io
If `isp` is not set, the output is just a string containing the client's IP address.
Note: if your server is behind some proxy, firewall, VPN, etc., the client's IP address may not be detected properly. If this happens, you must analyze the traffic coming from the client to find the name of the HTTP header that contains the original IP address. `getIP.php` contains some of these headers but not all of them.
#### CORS headers
All these files will send the following CORS headers if the GET parameter `cors=true` is passed to them:
The looks of this image can be customized by editing the variables in this file.
#### `idObfuscation.php`
Contains the implementation of ID obfuscation and deobfuscation.
See the code for the implementation details, it's basically a bunch of bitwise operations.
#### `stats.php`
Simple UI to display and search test results. Not required to run the test.
## Alternative backends
If for some reason you can't or don't want to use PHP, the speedtest can run with other backends, or even no backend (with limited functionality).
You will need replacements for `backend/garbage.php` and `backend/empty.php` and optionally `backend/getIP.php`, and the test needs to know where to find them:
Your replacement must simply respond with a HTTP code 200 and send nothing else. You may want to send additional headers to disable caching. The test assumes that `Connection:keep-alive` is sent by the server.
You will be able to run the download and upload test, but no IP, ISP and distance detection, no telemetry and results sharing, and only a single point of test.
* A replacement for `garbage.php`: a large incompressible file, like [this](http://downloads.fdossena.com/geth.php?r=speedtest-bigfile). We'll call this `backend/garbage.dat`
* A replacement for `empty.php`: an empty file will do. We'll call this `backend/empty.dat`
Now you need to configure the test to use them. Look for `s=new Speedtest()` and right below it, put the following:
```js
s.setParameter("url_dl","backend/garbage.dat");
s.setParameter("url_ul","backend/empty.dat");
s.setParameter("url_ping","backend/empty.dat");
s.setParameter("test_order","P_D_U");
```
This will point to our static files and set the test to only do ping/jitter, download and uplod tests.
These are the most common issues reported by users, and how to fix them. If you still need help, contact me at [info@fdossena.com](mailto:info@fdossena.com).
Are garbage.php and empty.php (or your replacements) reachable?
Press F12, select network and start the test. Do you see errors? (cancelled requests are not errors)
If a small download starts, open it in a text editor. Does it say it's missing openssl_random_pseudo_bytes()? In this case, install OpenSSL (this is usually included when you install Apache and PHP on most distros).
#### Upload test is inaccurate, and/or I see lag spikes
Check your server's maximum POST size, make sure it's at least 20Mbytes, possibly more
#### Download and/or upload results are slightly too optimistic
The test was fine tuned to run over a typical IPv4 internet connection. If you're using it under different conditions, see the `overheadCompensationFactor` parameter.
#### All tests are wrong, give extremely high results, browser lags/crashes, ...
You're running the test on localhost, therefore it is trying to measure the speed of your loopback interface. The test is meant to be run over an Internet connection, from a different machine.
#### The results sharing just generates a blank image
If the image doesn't display and the browser displays a broken image icon, FreeType2 is not installed or configured properly.
If the image is blank, this usually happens because PHP can't find the font files inside the `results` folder. You can fix your PHP config or edit `results/index.php` and use absolute paths for the fonts. This is a [known issue with PHP](http://php.net/manual/en/function.imagefttext.php) and no real solution is known.
#### My server is behind Cloudflare and I can't reach full speed on some of the tests
This is not a speedtest related issue, as it can be replicated in virtually any HTTP file upload/download.
Go to your domain's DNS settings and change "DNS and HTTP proxy (CDN)" to "DNS only", and wait for the settings to be applied (can take a few minutes).
* The ping/jitter test is measured by seeing how long it takes for an empty XHR to complete. It is not an acutal ICMP ping. Different browsers may also show different results, especially on very fast connections on slow devices.
* The upload test is not precise on very fast connections with high latency (will probably be fixed by Edge 17)
* On IE11, a same origin policy error is erroneously triggered under unknown conditions. Seems to be related to running the test from unusual URLs like a top level domain (for instance http://abc/speedtest). These are bugs in IE11's implementation of the same origin policy, not in the speedtest itself.
* On IE11, under unknown circumstances, on some systems the test can only be run once, after which speedtest_worker.js will not be loaded by IE until the browser is restarted. This is a rare bug in IE11.
* On some Linux systems with hardware acceleration turned off, the page rendering makes the browser lag, reducing the accuracy of the ping/jitter test, and potentially even the download and upload tests on very fast connections.
If you made some changes that you think should make it into the main project, send a Pull Request on GitHub, or contact me at [info@fdossena.com](mailto:info@fdossena.com).
We don't require you to use a specific coding convention, write the code however you want and we'll change the formatting if necessary.
Donations are also appreciated: you can donate with [PayPal](https://www.paypal.me/sineisochronic) or [Liberapay](https://liberapay.com/fdossena/donate).