PrivacyTests.org
News
About
Github icon
Twitter icon
Mastodon icon
LinkedIn icon
Email icon

News

Issue 85: 2024-04-12

New browser versions

In this issue, I have added Cromite to the Android browsers tested.

On iOS:

On Android:

Issue 84: 2024-04-01

New browser versions

On Desktop:

On iOS:

On Android:

Issue 83: 2024-03-15

New browser versions

On Desktop:

On iOS:

On Android:

Issue 82: 2024-03-01

New browser versions

On Desktop:

On iOS:

On Android:

Issue 81: 2024-02-14

New browser versions

On Desktop:

On iOS:

On Android:

Issue 80: 2024-02-01

New browser versions

On Desktop:

On iOS:

On Android:

New tests

This issue includes DNS privacy tests for the first time.

Issue 79: 2024-01-18

New browser versions

On Desktop:

On iOS:

On Android:

Issue 78: 2024-01-05

New browser versions

On Desktop:

On iOS:

On Android:

Issue 77: 2023-12-20

New browser versions

On Desktop:

On iOS:

On Android:

Issue 76: 2023-12-09

New browser versions

On Desktop:

On iOS:

On Android:

Issue 75: 2023-11-28

New browser versions

On Desktop:

On iOS:

On Android:

Issue 74: 2023-11-08

New browser versions

On Desktop:

On iOS:

On Android:

Issue 73: 2023-10-21

New browser versions

On Desktop:

On iOS:

On Android:

Issue 72: 2023-10-10

New browser versions

On Desktop:

On iOS:

On Android:

Issue 71: 2023-09-26

New browser versions

On Desktop:

On iOS:

On Android:

Issue 70: 2023-09-17

New browser versions

On Desktop:

On iOS:

On Android:

Issue 69: 2023-09-07

New test

We are now testing whether Encrypted Client Hello has been enabled by default. Short answer: not in any browsers yet.

New browser versions

On Desktop:

On iOS:

On Android:

Issue 68: 2023-08-28

New browser versions

On Desktop:

On iOS:

On Android:

Issue 67: 2023-08-20

New browser versions

On Desktop:

On iOS:

On Android:

Issue 66: 2023-08-02

New browser versions

On Desktop:

On iOS:

On Android:

Issue 65: 2023-07-23

New browser versions

On Desktop:

On iOS:

On Android:

Issue 64: 2023-07-16

New browser versions

On Desktop:

On iOS:

On Android:

Issue 63: 2023-07-08

New browser versions

On Desktop:

On iOS:

On Android:

New tests

In this weeks' issue, we have expanded the cross-session tracking tests to Desktop builds. LibreWolf, Mullvad, and Tor Browsers show especially strong protection against tracking between browser sessions.

Issue 62: 2023-07-02

New browser versions

On Desktop:

On iOS:

On Android:

New tests

In this week's issue, we have expanded the cross-session tracking tests to examine first-party tracking and third-party tracking in Nightly browser builds. In general, we see that websites and trackers are mostly able to track user across sessions, except if you are using Tor Browser, which deletes all history every time you quit.

Issue 61: 2023-06-23

New browser versions

On Desktop:

On iOS:

On Android:

New tests

This week's issue introduces cross-session tracking tests. We begin by testing Desktop Nightly browser builds to examine whether data is leaked across browser sessions so that a website can re-identify you when you visit a second time.

Issue 60: 2023-06-13

New browser versions

On Desktop:

On iOS:

On Android:

Issue 59: 2023-06-02

New browser versions

On Desktop:

On iOS:

On Android:

Issue 58: 2023-05-26

New browser versions

On iOS:

On Android:

Issue 57: 2023-05-20

New browser versions

On Desktop:

On iOS:

On Android:

Issue 56: 2023-05-13

New browser versions

On Desktop:

On iOS:

On Android:

Issue 55: 2023-05-06

New browser versions

On Desktop:

On iOS:

On Android:

Issue 54: 2023-04-28

New browser versions

On iOS:

On Android:

Issue 53: 2023-04-20

New browser versions

On Desktop:

On iOS:

On Android:

Issue 52: 2023-04-14

New browser versions

On Desktop:

On iOS:

On Android:

Issue 51: 2023-04-07

New browser versions

On Desktop:

On iOS:

On Android:

Issue 50: 2023-04-01

On Desktop:

On iOS:

On Android:

Issue 49: 2023-03-24

New browser versions

On Desktop:

On iOS:

On Android:

Issue 48: 2023-03-17

New browser versions

On Desktop:

On iOS:

On Android:

Issue 47: 2023-03-11

New browser versions

On Desktop:

On iOS:

On Android:

Issue 46: 2023-03-03

New browser versions

On iOS:

On Android:

Issue 45: 2023-02-23

New browser versions

On Desktop:

On iOS:

On Android:

Issue 44: 2023-02-18

On Desktop:

On Android:

Issue 43: 2023-02-03

New browser versions

On Desktop:

On iOS:

On Android:

Issue 42: 2023-01-27

New browser versions

On Desktop:

On iOS:

On Android:

Issue 41: 2023-01-19

New browser versions

On Desktop:

On iOS:

On Android:

Issue 40: 2023-01-08

New browser versions

On iOS:

On Android:

Issue 39: 2022-12-23

New browser versions

On Desktop:

On iOS:

On Android:

Issue 38: 2022-12-08

New browser versions

On Desktop:

On iOS:

On Android:

Issue 37: 2022-11-25

New browser versions

On Desktop:

On iOS:

On Android:

Issue 36: 2022-11-10

New browser versions

On Desktop:

On iOS:

On Android:

Issue 35: 2022-10-28

New browser versions

On Desktop:

On iOS:

On Android:

What's new

DuckDuckGo Desktop Beta (on MacOS) is now being tested -- results are shown in the Nightly section.

We see on Safari Desktop that the favicon cache is now partitioned! In Safari, Blob URLs is the only remaining API we test that still leaks data across websites.

In addition, Brave Nightly is now passing screen fingerprinting.

Issue 34: 2022-10-11

New browser versions

On Desktop:

On iOS:

On Android:

Response to Vivaldi

Last week's response to Vivaldi's claims is here.

Issue 33: 2022-09-20

New browser versions

On Desktop:

On iOS:

On Android:

Issue 32: 2022-09-06

New browser versions

On Desktop:

On iOS:

On Android:

Issue 31: 2022-08-23

New browser versions

On Desktop:

On iOS:

On Android:

Issue 30: 2022-08-08

Today, for the first time, Brave is now passing all State Partitioning tests. Congratulations to the team at Brave who worked on this!

New browser versions

On Desktop:

On iOS:

On Android:

Issue 29: 2022-07-26

New browser versions

On Desktop:

On iOS:

On Android:

Issue 28: 2022-07-11

New browser versions

On Desktop:

On iOS:

On Android:

Issue 27: 2022-06-30

Firefox improvement

For the first time, fresh profiles of Firefox are now passing (nearly) all State Partitioning tests, thanks to the worldwide rollout of Total Cookie Protection. Congratulations to the Firefox team! I am informed that existing profiles will also receive Total Cookie Protection in the next few months.

New browser versions

On Desktop:

On iOS:

On Android:

Issue 26: 2022-06-16

New browser versions

On Desktop:

On iOS:

On Android:

Issue 25: 2022-06-05

New browser versions

On Desktop:

On iOS:

On Android:

Issue 24: 2022-05-25

A faulty test fixed

It was brought to my attention that the "Tracker content blocking" test for Chartbeat was incorrectly reporting a "fail" for the DuckDuckGo Android browser. DuckDuckGo browser blocks third-party Chartbeat tracking scripts, but then provides the host page with a surrogate script to prevent breakage of the page's functionality. The original design of my test did not take into account this kind of surrogate, and so was incorrectly concluding that the original tracking script had been loaded into the page. I have now enhanced the test so it detects the presence of this surrogate and reports a "pass" for DuckDuckGo. Thanks to Peter Dolanjski for informing me of this problem.

New browser versions

On Desktop:

On iOS:

On Android:

Issue 23: 2022-05-16

New Desktop browser versions are:

New iOS browser versions are:

New Android browser versions are:

Issue 22: 2022-05-03

New test

This issue includes a new test for whether the Cookie Store API can be used to track users across sites in each browser. Thanks to Steven Englehardt for creating this test!

Updated browsers

New Desktop browser versions are:

New iOS browser versions are:

New Android browser versions are:

Issue 21: 2022-04-22

New Desktop browser versions are:

New iOS browser versions are:

New Android browser versions are:

Issue 20: 2022-04-07

New browser

In this issue I have added Mull to the set of Android browsers.

System font fingerprinting in Brave

Brave 1.39 (currently Nightly) has introduced a new protection against system font fingerprinting. It works by randomizing the user-installed fonts that are exposed to a web page. I am investigating how to test this new protection, so no "pass" or "fail" decision has yet been made.

New browser versions

New Desktop browser versions are:

New iOS browser versions are:

New Android browser versions are:

Issue 19: 2022-03-22

Correction

The Brave team reported a bug that resulted in incorrect results for the Alt-Svc test on the Brave browser. Apologies for the bug; I have corrected the issue. Thanks to Aleksey Khoroshilov and Pete Snyder for alerting me to the issue.

New browser

In this issue, we have added Firefox Focus to the set of Android browsers.

New browser versions

New iOS browser versions are:

New android browser versions are:

Issue 18: 2022-03-11

New browser versions

New desktop browser versions are:

New Android browser versions:

New iOS browser version:

Issue 17: 2022-03-04

Today I am publishing a set of new "tracking cookie protection" tests for desktop browsers. In these tests, we check whether the browser allows cookies from 19 of the top tracking domains to be shared across websites. The test works as follows:

  1. A web page from test site A is loaded with third-party tracking subresources, one from each tracking domain. A mitm proxy is used to inject a "Set-Cookie" header for each tracker.
  2. A second web page from test site B is loaded, with the same set of tracking subresources. The MITM proxy is again used to test whether it can read back the same cookies that were set for those tracking domains in step 1.

New browser versions

Desktop versions:

New Android browser versions:

New iOS browser versions:

Issue 16: 2022-02-16

I have expanded general cookie testing to examine both cross-site tracking via HTTP cookies and cross-site tracking via JavaScript cookies (aka document.cookie).

New versions

New desktop browser versions are:

New Android browser version:

New iOS browser versions:

Issue 15: 2022-02-03

After a brief pause to investigate an inconsistency in test results, we are back with Issue 15:

New desktop browser versions are:

On Android, one browser updated:

On iOS, updates are:

Investigation of inconsistency in four cache partitioning tests

Over the past week, I investigated puzzling behavior in four partitioning tests: CSS cache, font cache, image cache, and prefetch cache. Chromium-based browsers were passing these privacy tests, but, surprisingly, running the same tests manually or via a different testing framework resulted in failures. I wanted to understand why I was getting these inconsistent results, to make sure the published results are correct going forward.

Whether these tests passed or failed (i.e, isolation or sharing of data between websites) turned out to depend on how two pages from different websites were loaded. If the two pages are loaded completely independently, we see isolation, but if one page is loaded in a child tab of the other page, or if one page navigates to a second page, we see that the two pages can share cache data. That indicates that Chromium browsers are weakly isolating these caches, but not isolating them under all circumstances.

I decided to take the more stringent testing approach, on the principle that browsers should always isolate websites' data from one another except under user consent. So in this issue, the testing framework has been updated such that we see these tests newly failing for several Chromium-based browsers.

Thanks to Steven Englehardt for alerting me to this problem and providng helpful guidance.

Issue 14: 2022-01-21

This week, Opera Desktop has updated to version 83.0.

On Android, new browser versions are:

Issue 13: 2022-01-14

This week, new desktop browser versions include:

On iOS, new browser versions are:

And on Android, we have:

Issue 12: 2022-01-07

This week, new desktop browser versions include:

Issue 11: 2021-12-31

New browsers

This week we have added the privacy-oriented Bromite browser to our Android tests, and Ungoogled Chromium to our desktop tests.

Issue 10.1: 2021-12-26

Issue 10.1 fixes a problem in Issue 10 where Alt-Svc and H3 connection tests weren't operating properly.

Issue 10: 2021-12-24

(Desktop, Private modes, Android, iOS, Nightly, Nightly private modes)

Introducing LibreWolf tests

In Issue 10, we have added LibreWolf to the set of tested browsers. LibreWolf is a Firefox-based browser with some unique default privacy features not found in other browsers.

Updated layout

We have separated out Private Modes (aka Private Browsing, Incognito etc.) into their own tables for Desktop and Nightly browsers.

New browser versions

Since last week, some browser versions have updated:

Desktop versions haven't updated this week.

Added test

I have separated the Global Privacy Control test into "GPC enabled first-party" and "GPC enabled third-party."

Issue 9: 2021-12-16

(Desktop, Nightly, Android, iOS)

Introducing mobile web browser testing

This issue adds two additional platforms for browser testing: Android and iOS. The new browsers are:

A new suite of tests for tracker content blocking

Some web browsers maintain a blocklist of tracking domains. Third-party content (such as tracking pixels and tracking scripts) from these domains are blocked by the browser so that they are not loaded into the page. To see which browsers carry out this form of blocking, and what domains they block, Issue 9 introduces tracker content blocking tests. For 20 of the most common tracking domains reported by whotracks.me, the tests attempt to load a tracking script or image. A browser passes the test if it blocks the script or image from being loaded.

In this first run: Brave, DuckDuckGo, Firefox Private Mode and Firefox Focus were found to do substantial tracking content blocking.

Known issue

Again we have skipped testing of Firefox Nightly because of the browser crash.

Issue 8 (Desktop, Nightly: 2021-12-09)

New browser versions

Since Issue 7, Firefox has updated to version 95.0.

Known issue

Because of a crash in Firefox Nightly, it is not included in this week's Nightly browser testing.

Issue 7 (2021-12-02)

New browser versions

Since Issue 6, Opera has updated to 82.0 and Vivaldi to 5.0.

Known issue

Because of a crash in Firefox Nightly, it is not included in this week's testing.

Issue 6 (2021-11-24)

New browser version

Since Issue 5, Edge has updated to version 96.0

A privacy improvement in Brave

Brave has introduced an important new partitioning behavior. HTTP1, HTTP2, and HTTP3 connections are now partitioned by first party. That means your web connections can no longer be used to correlate your visits between different websites.

Thanks and congratulations to the Brave team for this fix!

Issue 5 (2021-11-17)

New browser versions

Since Issue 4, three browsers have updates:

Issue 4 (2021-11-09)

Testing of nightly builds added

I have now added testing of the Nightly build channel (or the nearest equivalent) for all monitored desktop browers. These include:

These tests give a preview of future privacy developments in these browsers. And I hope it offers faster feedback for browser development teams as they land patches for new privacy protections.

New browser versions

Since Issue 3, Firefox has updated to v. 94.0.

Issue 3 (2021-11-02)

New browser versions

Since Issue 2, new browser releases include Chrome 95.0, Edge 95.0, and Safari 15.1.

New tests, new results

Three new tests have been added. These are:

  1. Alt-Svc. When you visit a website for the first time, an Alt-Svc header may be sent to your browser to indicate that the same website can be fetched in another location or using another protocol. For subsequent, visits, the browser may use that alternate location or protocol instead of the one it originally used on the first connection. A common use of Alt-Svc is for the website suggest to the browser to upgrade the connection from HTTP/2 to HTTP/3. Unfortunately this protocol can leak information about which websites you have visited in the past and even be abused to track you across sites.

  2. Stream isolation. In Tor Browser, every website gets its own circuit such that all first-party requests and third-party embedded requests for that website are on a separate stream from those of any other website. This helps to reduce the ability of adversaries to correlate a browser's connection to two different websites.

  3. System Font fingerprinting. If you install a new font on your computer, most browsers will helpfully use that font if it is ever requested by a website you visit. Unfortunately, that reveals to the website that you have installed the font. That information leak turns out to be quite an important source of fingerprinting entropy, making it easier to track you on the web Today's results show that Safari and Tor Browser protect against this type of fingerprinting.

Issue 2 (2021-10-25)

Correction

The first issue of PrivacyTests.org had an important error in the results, incorrectly indicating that Safari does not stop tracking via third-party cookies. Safari cookie protections were assigned an when it should have been a . This incorrect results seems to have happened because the Selenium Webdriver library I had been using to launch and control the various web browsers likely disables Safari's Intelligent Tracking Protection feature. This new issue of PrivacyTests.org results shows the correct for cookie protections in Safari.

My apologies for the error. Thanks to John Wilander and Steven Englehardt for bringing this issue to my attention.

Code updates

Major updates have been made to the testing code. Because of the error mentioned above, I decided to discontinue the use of Selenium Webdriver altogether in the PrivacyTests.org. Instead, the code has now been extensively rewritten to launch each web browser by executing a shell command, and to direct the web browsers to testing pages via shell commands as well. This new approach has the advantage of more closely mimicking a web browser in its "natural" state. The new code also makes it possible to launch Safari in both standard windows and Private Windows.

Follwing this rewrite, the PrivacyTests.org testing code now runs on macOS only. I plan to extend the new code to be compatible with Linux and Windows in the future.

New results

Due to popular request, I have added Vivaldi (currently version 4.3) to the roster of tested browsers. In addition, since Issue 1, some browsers have been updated to the latest release versions, including Brave 1.31, Edge 95.0, and Safari 15.0.

Thank you

Thanks to everyone who gave feedback following the launch. Everyone's comments and suggestions for future improvements are much appreciated!

Issue 1 (2021-10-13)

PrivacyTests.org went live for the first time, presenting desktop browser privacy test results for Brave 1.30, Chrome 94.0, Edge 94.0, Firefox 93.0, Opera 80.0, Safari 14.1, and Tor 10.5.