Planet Tor

@kushal August 17, 2022 - 11:28 • 10 hours ago
johnnycanencrypt 0.7.0 released

Today I released Johnnycanencrypt 0.7.0. It has breaking change of some function names.

  • create_newkey renamed to create_key
  • import_cert renamed to import_key

But, the major work done are in few different places:

  • Handling errors better, no more normal Rust panics, instead providing better Python exceptions as CryptoError.
  • We can now sign bytes/files in both detached & in normal compressed binary form.
  • Signature can be done via smartcards, and verification works as usual.

In the Github release page you can find an OpenPGP signature, which you can use to verify the release. You can also verify via sigstore.

SIGSTORE_LOGLEVEL=debug python -m sigstore verify --cert-email mail@kushaldas.in --cert-oidc-issuer https://github.com/login/oauth johnnycanencrypt-0.7.0.tar.gz
DEBUG:sigstore._cli:parsed arguments Namespace(subcommand='verify', certificate=None, signature=None, cert_email='mail@kushaldas.in', cert_oidc_issuer='https://github.com/login/oauth', rekor_url='https://rekor.sigstore.dev', staging=False, files=[PosixPath('johnnycanencrypt-0.7.0.tar.gz')])
DEBUG:sigstore._cli:Using certificate from: johnnycanencrypt-0.7.0.tar.gz.crt
DEBUG:sigstore._cli:Using signature from: johnnycanencrypt-0.7.0.tar.gz.sig
DEBUG:sigstore._cli:Verifying contents from: johnnycanencrypt-0.7.0.tar.gz
DEBUG:sigstore._verify:Successfully verified signing certificate validity...
DEBUG:sigstore._verify:Successfully verified signature...
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): rekor.sigstore.dev:443
DEBUG:urllib3.connectionpool:https://rekor.sigstore.dev:443 "POST /api/v1/index/retrieve/ HTTP/1.1" 200 85
DEBUG:urllib3.connectionpool:https://rekor.sigstore.dev:443 "GET /api/v1/log/entries/362f8ecba72f4326972bc321d658ba3c9197b29bb8015967e755a97e1fa4758c13222bc07f26d27c HTTP/1.1" 200 None
DEBUG:sigstore._verify:Successfully verified Rekor entry...
OK: johnnycanencrypt-0.7.0.tar.gz

I took 8 months for this release, now time to write some tools to use it in more places :)

...
@blog August 9, 2022 - 00:00 • 9 days ago
New Alpha Release: Tor Browser 12.0a1 (Windows, macOS, Linux)

Tor Browser 12.0a1 is now available from the Tor Browser download page and also from our distribution directory.

Tor Browser 12.0a1 updates Firefox on Windows, macOS, and Linux to 91.12.0esr.

We use the opportunity as well to update various other components of Tor Browser:

  • Tor Launcher 0.2.38
  • OpenSSL 1.1.1q
  • Go 1.17.13

This version includes important security updates to Firefox.

The full changelog since Tor Browser 11.5a13 is:

...
@blog August 3, 2022 - 00:00 • 15 days ago
Arti 0.6.0 is released: bugfixes, error improvements, and preparation.

Arti is our ongoing project to create a working embeddable Tor client in Rust. It’s not ready to replace the main Tor implementation in C, but we believe that it’s the future.

Right now, our focus is on making Arti production-quality, by stress-testing the code, hunting for likely bugs, and adding missing features that we know from experience that users will need. We're going to try not to break backward compatibility too much, but we'll do so when we think it's a good idea.

What's new in 0.6.0?

For a complete list of changes, have a look at the CHANGELOG.

In this release, we reworked a large number of internal APIs to simplify and correct our code, to provide better error messages, and provide more of Tor's cryptographic handshakes.

We also fixed major compatibility issues on Windows and iOS.

There are also a bunch of smaller features, bugfixes, and infrastructure improvements; again, see the CHANGELOG for a more complete list.

And what's next?

In the short term, we're working for feature-parity with Tor in netflow resistance and congestion control. We already have some backend code, and hope to land the remainder soon.

Beyond that, between now and our 1.0.0 milestone in September, we're aiming to make Arti a production-quality Tor client for direct internet access. (Onion services aren't funded yet, but we hope to change that soon.)

To do so, we need to bring Arti up to par with the C tor implementation in terms of its CPU usage, and security features. You can follow our progress on our 1.0.0 milestone.

We still plan to continue regular releases between now and then.

Here's how to try it out

We rely on users and volunteers to find problems in our software and suggest directions for its improvement. Although Arti isn't yet ready for production use, you can test it as a SOCKS proxy (if you're willing to compile from source) and as an embeddable library (if you don't mind a little API instability).

Assuming you've installed Arti (with cargo install arti, or directly from a cloned repository), you can use it to start a simple SOCKS proxy for making connections via Tor with:

$ arti proxy -p 9150

and use it more or less as you would use the C Tor implementation!

(It doesn't support onion services yet. If compilation doesn't work, make sure you have development files for libsqlite installed on your platform.)

If you want to build a program with Arti, you probably want to start with the arti-client crate. Be sure to check out the examples too.

For more information, check out the README file. (For now, it assumes that you're comfortable building Rust programs from the command line). Our CONTRIBUTING file has more information on installing development tools, and on using Arti inside of Tor Browser. (If you want to try that, please be aware that Arti doesn't support onion services yet.)

When you find bugs, please report them on our bugtracker. You can request an account or report a bug anonymously.

And if this documentation doesn't make sense, please ask questions! The questions you ask today might help improve the documentation tomorrow.

Call for feedback

Our priority for the coming months is to make Arti a production-quality Tor client, for the purposes of direct connections to the internet. (Onion services will come later.) We know some of the steps we'll need to take to get there, but not all of them: we need to know what's missing for your use-cases.

Whether you're a user or a developer, please give Arti a try, and let us know what you think. The sooner we learn what you need, the better our chances of getting it into an early milestone.

Acknowledgments

Thanks to everyone who has contributed to this release, including Arturo Marquez, Dimitris Apostolou, feelingnothing, Jim Newsome, Richard Pospesel, spongechameleon, Trinity Pointard, and Yuan Lyu.

And thanks, of course, to Zcash Community Grants (formerly Zcash Open Major Grants (ZOMG)) for funding this project!

...
@blog August 1, 2022 - 00:00 • 17 days ago
Help Smash Tor Bugs!

Smashing Tor Bugs in 2022

Last year, your support of the Bug Smash Fund helped us solve 241 tickets related to Tor bugs and maintenance.

From smashing bugs related to anti-censorship features for censored users, resolving issues in Tor Browser, and conducting sysadmin maintenance, to squashing bugs on the network, resolving errors on metrics.torproject.org, and making documentation updates, you've powered the behind-the-scenes work that keeps Tor safe and strong.

If we assume each closed ticket required just 30 minutes of work to fix the bug, you made 120 hours (about one month) of Tor improvement work possible.

Today, we need your help to smash more bugs.

In 2022, we've been hard at work. Connection Assist in Tor Browser 11.5 has made it much easier for people to route around censorship against Tor. User support channels on Telegram, and in Russia, have expanded Tor access in places where its desperately needed. Onion service admins have received new DoS protections for their services. Congestion control has arrived in Tor Browser stable. Arti 0.5.0 has brought more stability to our Tor re-write in Rust. These improvements need maintenance over time—and keeping these tools running smoothly is just as critical as building new features.

This year, we must raise $75,000 to ensure we can smash Tor bugs and conduct needed maintenance.

Last year, the average Bug Smash Fund donor gave $60. Can you help keep Tor users safe from bugs and connected to the open internet with a donation of $60? Without Tor, many of our users would not have a safe option for getting online. Helping us smash bugs keeps Tor's most vulnerable users safe, like this user from Turkey:

What I can get with Tor, I could not get anywhere in [else] in Turkey: information, news, banned articles, and much more... I couldn't safely [use the] internet without Tor. - Anonymous Tor User

Every donation made to the Tor Project in the month of August will go towards the Bug Smash Fund.

Make your contribution today and keep Tor strong. Thank you!

2021 Bug Smash Results

2021 Bug Smash Results

Below we'll highlight areas of work—and related solved tickets—made possible with your support of the Bug Smash Fund.

👾 Bugs related to anti-censorship tools (23 tickets)

👾 Bugs related to metrics and network health (13 tickets)

👾 Bugs related to the Tor network (97 tickets)

👾 Bugs releated to Tor applications (44 tickets)

📝 Documentation projects and updates (35 tickets)

🛠️ System & service administration projects (3 tickets)

...
@ooni July 29, 2022 - 00:00 • 20 days ago
New Test Lists Editor: Contribute websites for censorship testing
Today the Open Observatory of Network Interference (OONI) team is excited to announce the public launch of our new Test Lists Editor! This platform enables the public to review and contribute to the lists of websites (“test lists”) that are tested for censorship by OONI Probe users around the world. Help the internet freedom community discover website blocks around the world by contributing through the new Test Lists Editor! ...
@blog July 26, 2022 - 00:00 • 23 days ago
New Release: Tor Browser 11.5.1 (Windows, macOS, Linux)

Tor Browser 11.5.1 is now available from the Tor Browser download page and also from our distribution directory.

Tor Browser 11.5.1 updates Firefox on Windows, macOS, and Linux to 91.12.0esr.

We would like to thank WofWca for sending us some patches for the preferences page.

The full changelog since Tor Browser 11.5 is:

...
@anarcat July 20, 2022 - 17:22 • 28 days ago
Relaying mail through debian.org

Back in 2020, I wrote this article about using DKIM to sign outgoing debian.org mail. This worked well for me for a while: outgoing mail was signed with DKIM and somehow was delivered. Maybe. Who knows.

But now we have a relay server which makes this kind of moot. So I have changed my configuration to use that relay instead of sending email on my own. It seems more reliable that mail seems to be coming from a real debian.org machine, so I'm hoping this will have better reputation than my current setup.

In general, you should follow the DSA documentation which includes a Postfix configuration. In my case, it was basically this patch:

diff --git a/postfix/main.cf b/postfix/main.cf
index 7fe6dd9e..eabe714a 100644
--- a/postfix/main.cf
+++ b/postfix/main.cf
@@ -55,3 +55,4 @@ smtp_sasl_security_options =
 smtp_sender_dependent_authentication = yes
 sender_dependent_relayhost_maps = hash:/etc/postfix/sender_relay
 sender_dependent_default_transport_maps = hash:/etc/postfix/sender_transport
+smtp_tls_policy_maps = hash:/etc/postfix/tls_policy
diff --git a/postfix/sender_relay b/postfix/sender_relay
index b486d687..997cce19 100644
--- /dev/null
+++ b/postfix/sender_relay
@@ -0,0 +1,2 @@
+# Per-sender provider; see also /etc/postfix/sasl_passwd.
+@debian.org    [mail-submit.debian.org]:submission
diff --git a/postfix/sender_transport b/postfix/sender_transport
index ca69bc7a..c506c1fc 100644
--- /dev/null
+++ b/postfix/sender_transport
@@ -0,0 +1,1 @@
+anarcat@debian.org     smtp:
diff --git a/postfix/tls_policy b/postfix/tls_policy
new file mode 100644
index 00000000..9347921a
--- /dev/null
+++ b/postfix/tls_policy
@@ -0,0 +1,1 @@
+submission.torproject.org:submission   verify ciphers=high

This configuration differs from the one provided by DSA because I already had the following configured:

sender_dependent_relayhost_maps = hash:/etc/postfix/sender_relay
smtp_sender_dependent_authentication = yes
smtp_sasl_auth_enable = yes
smtp_sasl_password_maps = hash:/etc/postfix/sasl_passwd
smtp_sasl_tls_security_options = noanonymous

I also don't show the patch on /etc/postfix/sasl_passwd for obvious security reasons.

I also had to setup a tls_policy map, because I couldn't use dane for all my remotes. You'll notice I also had to setup a sender_transport because I use a non-default default_transport as well.

It also seems like you can keep the previous DKIM configuration in parallel with this one, as long as you don't double-sign outgoing mail. Since this configuration here is done on my mail client (i.e. not on the server where I am running OpenDKIM), I'm not double-signing so I left the DKIM configuration alone. But if I wanted to remove it, the magic command is:

echo "del dkimPubKey" | gpg --clearsign | mail changes@db.debian.org
...
@kushal July 16, 2022 - 08:01 • 1 months ago
dgplug mailing list has a new home

We were using the mailman2 instance provided by Dreamhost for many years as the mailing list for dgplug. But, over the years many participants had trouble with receiving emails. In the last few years, most emails were landing in spam.

So, we took the chance to move to a new mailing list, and also started working on the site to have CoC properly defined. To make things easier, we will just follow the PSF Code of conduct https://www.python.org/psf/conduct/, most of our members are already parts of various upstream communities. So, this will be nothing new for them. We will be also updating our sites to add details of a separate team who will handle CoC violation reports.

Summer Training will start from 25th July, so remember to join in the new mailing list before that. See you all on IRC #dgplug channel on Libera server.

...
@blog July 14, 2022 - 00:00 • 1 months ago
New Release: Tor Browser 11.5

Tor Browser 11.5 is now available from the Tor Browser download page and also from our distribution directory. This new release builds upon features introduced in Tor Browser 10.5 to transform the user experience of connecting to Tor from heavily censored regions.

Update (2022-07-19): We're tracking two known issues affecting certain language versions of Tor Browser, and users who attempt to visit IP addresses that do not support HTTPS. Please see Known issues below for details.

What's new?

Automatic censorship detection and circumvention

We began reshaping the experience of connecting to Tor with the release of Tor Browser 10.5 last year, including the retirement of the Tor Launcher and the integration of the connection flow into the browser window. However, circumventing censorship of the Tor Network itself remained a manual and confusing process – requiring users to dive into Tor Network settings and figure out for themselves how to apply a bridge to unblock Tor. What's more, censorship of Tor isn't uniform – and while a certain pluggable transport or bridge configuration may work in one country, that doesn't mean it'll work elsewhere.

This placed the burden on censored users (who are already under significant pressure) to figure out what option to pick, resulting in a lot of trial, error and frustration in the process. In collaboration with the Anti-Censorship team at the Tor Project, we've sought to reduce this burden with the introduction of Connection Assist: a new feature that when required will offer to automatically apply the bridge configuration we think will work best in your location for you.

Connection assist

Connection Assist works by looking up and downloading an up-to-date list of country-specific options to try using your location (with your consent). It manages to do so without needing to connect to the Tor Network first by utilizing moat – the same domain-fronting tool that Tor Browser uses to request a bridge from torproject.org.

While Connection Assist has reached the milestone of its first stable release, this is only version 1.0, and your feedback will be invaluable to help us improve its user experience in future releases. Users from countries where the Tor Network may be blocked (such as Belarus, China, Russia and Turkmenistan) can test the most recent iteration of this feature by volunteering as an alpha tester, and reporting your findings on the Tor forum.

Redesigned Tor Network settings

Connection settings

We hope that the majority of our users living under extreme censorship will be able to connect to Tor at the press of a button, thanks to Connection Assist. However we know there will always be exceptions to that, and there are many users who prefer to configure their connection manually as well.

That's why we've invested time redesigning Tor Network settings too – featuring:

  • A brand new name: Tor Network settings is now called Connection settings. This change is intended to clarify exactly what settings you can find within this tab.
  • Connection statuses: Your last known connection status can now be found at the top of the tab, including the option to test your Internet connection without Tor, using moat, to help you untangle the source of your connection woes.
  • Streamlined bridge options: Gone is the long list of fields and options. Each method to add a new bridge has been tidied away into individual dialog menus, which will help support further improvements to come.
  • Connection Assist: When Tor Browser's connection to the Tor Network isn't reachable due to suspected censorship, an additional option to select a bridge automatically becomes available.
  • Brand-new bridge cards: Bridges used to be almost invisible, even when configured. Now, your saved bridges appear in a handy stack of bridge cards – including new options for sharing bridges too.

Bridge card diagram

This is the anatomy of a bridge card when expanded. In addition to copying and sharing the bridge line, each bridge also comes with a unique QR code that will be readable by Tor Browser for Android (and hopefully other Tor-powered apps too) in a future release – helping facilitate the transfer of a working bridge from desktop to mobile.

When you have multiple bridges configured the cards will collapse into a stack – each of which can be expanded again with a click. And when connected, Tor Browser will let you know which bridge it's currently using with the purple "✔ Connected" pill. To help differentiate between your bridges without needing to compare long, unfriendly bridge lines, we've introduced bridge-moji: a short, four emoji visualization you can use to identify the right bridge at a glance.

Lastly, help links within Connection settings now work offline. To recap – there are two types of help links in Tor Browser's settings: those that point to support.mozilla.org, and those that point to tb-manual.torproject.org (i.e. the Tor Browser Manual). However, since web-based links aren't very useful when you're troubleshooting connection issues with Tor Browser, the manual is now bundled in Tor Browser 11.5 and is available offline. In addition to the help links within Tor Browser's settings, the manual can be accessed via the Application Menu > Help > Tor Browser Manual, and by entering "about:manual" into your browser's address bar too.

HTTPS-Only Mode, by default

HTTPS-Only Mode

HTTPS-Everywhere is one of two extensions that previously came bundled in Tor Browser, and has led a long and distinguished career protecting our users by automatically upgrading their connections to HTTPS wherever possible. Now, HTTPS is actually everywhere, and all major web browsers include native support to automatically upgrade to HTTPS. Firefox – the underlying browser on which Tor Browser is based – calls this feature HTTPS-Only Mode.

Starting in Tor Browser 11.5, HTTPS-Only Mode is enabled by default for desktop, and HTTPS-Everywhere will no longer be bundled with Tor Browser.

Why now? Research by Mozilla indicates that the fraction of insecure pages visited by the average users is very low – limiting the disruption caused to the user experience. Additionally, this change will help protect our users from SSL stripping attacks by malicious exit relays, and strongly reduces the incentive to spin up exit relays for Man-in-the-Middle attacks in the first place.

You may or may not know that HTTPS-Everywhere also served a second purpose in Tor Browser, and was partly responsible for making SecureDrop's human-readable onion names work. Well, SecureDrop users can rest assured that we've patched Tor Browser to ensure that human-readable onion names still work in HTTPS-Everywhere's absence.

Note: Unlike desktop, Tor Browser for Android will continue to use HTTPS-Everywhere in the short term. Please see our separate update about Android below.

Improved font support

More fonts

One of Tor Browser's many fingerprinting defenses includes protection against font enumeration – whereby an adversary can fingerprint you using the fonts installed on your system. To counter this, Tor Browser ships with a standardized bundle of fonts to use in place of those installed on your system. However some writing scripts did not render correctly, while others had no font available in Tor Browser at all.

To solve this issue and expand the number of writing systems supported by Tor Browser, we've bundled many more fonts from the Noto family in this release. Naturally, we have to find a balance between the number of fonts Tor Browser supports without increasing the size of the installer too much, which is something we're very conscious of. So if you spot a language whose characters don't render correctly in Tor Browser, please let us know!

Tor Browser for Android

You have no doubt noticed that the features announced above are all for desktop. So, we wanted to share a little update about where we're at with Android:

We know that Tor Browser for Android is quite behind desktop in terms of feature parity. The Tor Project has hit a few bumps in the road over the last couple of years that have delayed our releases, and led us to reassess our roadmap for Android. Since the beginning of the year our priorities for Android have been three-fold:

  1. Start releasing regular updates for Android again
  2. Fix the crashes that many Android users have experienced
  3. Begin catching up with Fenix (Firefox for Android) releases

Since then, Android has averaged one stable update per month, crash reports are down significantly thanks to the patch issued in fenix#40212, and downloads are working again due to the fixes in fenix#40192 and android-components#40075. However we still have work to do to catch up with Fenix, and upgrading Tor Browser to Fenix v102 will be our priority for the next few months.

We've also taken steps to expand the team's capacity in order to dedicate more resources to Android, keep the application stable, and help us bring some of these features described above to Android in the future too.

Thank you for your patience and support!

Known issues

Tor Browser 11.5 comes with a number of known issues:

Bug torbrowser#40159: Bridge cards aren't displaying, and toggle themselves off

We're aware of an issue affecting certain language-versions of Tor Browser that's preventing bridge cards from rendering within Connection settings, even if a bridge has been configured. Furthermore, bridges can appear to toggle themselves on and off too.

Our initial testing indicates that this bug is limited to the UI only, and Tor Browser will remain connected to the bridge you have input regardless. We're working on a fix for this issue as a priority.

Bug torbrowser#41050: "Continue to HTTP Site" button doesn't work on IP addresses

HTTPS-Only Mode will alert you whenever a HTTP connection cannot be upgraded. Normally this alert can by bypassed using the "Continue to HTTP Site" button, which grants an exception for the site in question. However this button does not work when visiting an IP address directly, and an exception cannot be granted by other means.

A fix for this issue will be coming soon. Should you need to visit an IP address over HTTP in the meantime, we recommend against turning HTTPS-Only Mode off. Instead, consider downgrading temporarily to Tor Browser 11.0.15 until fixed.

Send us your feedback

If you find a bug or have a suggestion for how we could improve this release, please let us know. Thanks to all of the teams across Tor, and the many volunteers, who contributed to this release.

Full changelog

The full changelog since Tor Browser 11.0.15 is:

...
@blog July 12, 2022 - 00:00 • 1 months ago
New Release: Tails 5.2

Changes and updates

Included software

  • Update Tor Browser to 11.5.

  • Update Thunderbird to 91.11.0.

Fixed problems

For more details, read our changelog.

Known issues

None specific to this release.

See the list of long-standing issues.

Get Tails 5.2

To upgrade your Tails USB stick and keep your persistent storage

  • Automatic upgrades are available from Tails 5.0 or later to 5.2.

You can reduce the size of the download of future automatic upgrades by doing a manual upgrade to the latest version.

  • If you cannot do an automatic upgrade or if Tails fails to start after an automatic upgrade, please try to do a manual upgrade.

To install Tails on a new USB stick

Follow our installation instructions:

The Persistent Storage on the USB stick will be lost if you install instead of upgrading.

To download only

If you don't need installation or upgrade instructions, you can download Tails 5.2 directly:

What's coming up?

Tails 5.3 is scheduled for July 26.

Have a look at our roadmap to see where we are heading to.

Support and feedback

For support and feedback, visit the Support section on the Tails website.

...
@kushal July 5, 2022 - 15:41 • 1 months ago
Using sigstore-python to sign and verify your software release

Sigstore allows software developers to quickly sign and verify the software they release. Many of the bigger projects use hardware-based OpenPGP keys to sign and release. But the steps used to make sure that the end-users are correctly verifying those signatures are long, and people make mistakes. Also, not every project has access to hardware smartcards, air-gapped private keys etc. Sigstore solves (or at least makes it way easier) these steps for most developers. It uses existing known (right now only 3) big OIDC providers using which one can sign and verify any data/software.

For this blog post, I will use the python tool called sigstore-python.

The first step is to create a virtual environment and then install the tool.

$ python3 -m venv .venv
$ source .venv/bin/activate
$ python -m pip install -r install/requirements.txt

Next, we create a file called message.txt with the data. This can be our actual release source code tarball.

$ echo "Kushal loves Python!" > message.txt

Signing the data

The next step is to actually sign the file.

$ python -m sigstore sign message.txt 
Waiting for browser interaction...
Using ephemeral certificate:
-----BEGIN CERTIFICATE-----
...
-----END CERTIFICATE-----

Transparency log entry created at index: 2844439
Signature written to file message.txt.sig
Certificate written to file message.txt.crt

The command will open up the default browser, and we will have the choice to select one of the 3 following OIDC providers.

oidc providers

This will also create message.txt.crt & message.txt.sig files in the same directory.

We can use the openssl command to see the contents of the certificate file.

$ openssl x509 -in message.txt.crt -noout -text
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            3a:c4:2d:19:20:f0:bf:85:37:a6:01:0f:49:d1:b6:39:20:06:fd:77
        Signature Algorithm: ecdsa-with-SHA384
        Issuer: O = sigstore.dev, CN = sigstore-intermediate
        Validity
            Not Before: Jul  5 14:45:23 2022 GMT
            Not After : Jul  5 14:55:23 2022 GMT
        Subject: 
        Subject Public Key Info:
            Public Key Algorithm: id-ecPublicKey
                Public-Key: (384 bit)
                pub:
                    04:12:aa:88:fd:c7:1f:9e:62:78:46:2a:48:63:d3:
                    b6:92:8b:51:a4:eb:59:18:fb:18:a0:13:54:ac:d0:
                    a4:d8:20:ab:a3:f3:5e:f5:86:aa:34:9b:30:db:59:
                    1b:5c:3d:29:b1:5a:40:ff:55:2c:26:fc:42:58:95:
                    53:d6:23:e5:66:90:3c:32:8c:82:b7:fc:fd:f8:28:
                    2b:53:2d:5c:cb:df:2f:17:d0:f3:bc:26:d2:42:3d:
                    c0:b1:55:61:50:ff:18
                ASN1 OID: secp384r1
                NIST CURVE: P-384
        X509v3 extensions:
            X509v3 Key Usage: critical
                Digital Signature
            X509v3 Extended Key Usage: 
                Code Signing
            X509v3 Subject Key Identifier: 
                6C:F0:C0:63:B8:3D:BB:08:90:C3:03:45:FF:55:92:43:7D:47:19:38
            X509v3 Authority Key Identifier: 
                DF:D3:E9:CF:56:24:11:96:F9:A8:D8:E9:28:55:A2:C6:2E:18:64:3F
            X509v3 Subject Alternative Name: critical
                email:mail@kushaldas.in
            1.3.6.1.4.1.57264.1.1: 
                https://github.com/login/oauth
            CT Precertificate SCTs: 
                Signed Certificate Timestamp:
                    Version   : v1 (0x0)
                    Log ID    : 08:60:92:F0:28:52:FF:68:45:D1:D1:6B:27:84:9C:45:
                                67:18:AC:16:3D:C3:38:D2:6D:E6:BC:22:06:36:6F:72
                    Timestamp : Jul  5 14:45:23.112 2022 GMT
                    Extensions: none
                    Signature : ecdsa-with-SHA256
                                30:46:02:21:00:AB:A6:ED:59:3E:B7:C4:79:11:6A:92:
                                29:92:BF:54:45:6A:B6:1F:6F:1C:63:7C:D9:89:26:D4:
                                6B:EF:E3:3E:9F:02:21:00:AD:87:A7:BA:BA:7C:61:D2:
                                53:34:E0:D0:C4:BF:6A:6E:28:B4:02:82:AA:F8:FD:0B:
                                FB:3A:CD:B9:33:3D:F4:36
    Signature Algorithm: ecdsa-with-SHA384
    Signature Value:
        30:65:02:30:17:89:76:ef:a1:0e:97:5b:a3:fe:c0:34:13:36:
        3f:6f:2a:ba:e9:cd:bd:f2:74:d9:8c:13:2a:88:c9:96:b2:72:
        de:34:44:95:41:f8:b0:69:5b:f0:86:a7:05:cf:81:7f:02:31:
        00:d8:3a:12:89:39:4b:2c:ad:ff:5a:23:85:d9:c0:73:f0:b1:
        db:5c:65:f9:5d:ee:7a:bb:b8:08:01:44:7a:2e:9f:ba:2b:4b:
        df:6a:93:08:e9:44:2c:23:88:66:2c:f7:8f

Verifying the signature

We can verify the signature, just make sure that the certificate & signature files are in the same directory.

$ python -m sigstore verify message.txt 
OK: message.txt

Now, to test this with some real software releases, we will download the cosign RPM package and related certificate & signature files. The certificate in this case, is base64 encoded, so we decode that file first.

$ curl -sOL https://github.com/sigstore/cosign/releases/download/v1.9.0/cosign-1.9.0.x86_64.rpm
$ curl -sOL https://github.com/sigstore/cosign/releases/download/v1.9.0/cosign-1.9.0.x86_64.rpm-keyless.sig
$ curl -sOL https://github.com/sigstore/cosign/releases/download/v1.9.0/cosign-1.9.0.x86_64.rpm-keyless.pem
$ base64 -d cosign-1.9.0.x86_64.rpm-keyless.pem > cosign-1.9.0.x86_64.rpm.pem

Now let us verify the downloaded RPM package along with the email address and signing OIDC issuer URL. We are also printing the debug statements, so that we can see what is actually happening for verification.

$ SIGSTORE_LOGLEVEL=debug python -m sigstore verify --certificate cosign-1.9.0.x86_64.rpm.pem --signature cosign-1.9.0.x86_64.rpm-keyless.sig --cert-email keyless@projectsigstore.iam.gserviceaccount.com --cert-oidc-issuer https://accounts.google.com  cosign-1.9.0.x86_64.rpm

DEBUG:sigstore._cli:parsed arguments Namespace(subcommand='verify', certificate=PosixPath('cosign-1.9.0.x86_64.rpm.pem'), signature=PosixPath('cosign-1.9.0.x86_64.rpm-keyless.sig'), cert_email='keyless@projectsigstore.iam.gserviceaccount.com', cert_oidc_issuer='https://accounts.google.com', rekor_url='https://rekor.sigstore.dev', staging=False, files=[PosixPath('cosign-1.9.0.x86_64.rpm')])
DEBUG:sigstore._cli:Using certificate from: cosign-1.9.0.x86_64.rpm.pem
DEBUG:sigstore._cli:Using signature from: cosign-1.9.0.x86_64.rpm-keyless.sig
DEBUG:sigstore._cli:Verifying contents from: cosign-1.9.0.x86_64.rpm
DEBUG:sigstore._verify:Successfully verified signing certificate validity...
DEBUG:sigstore._verify:Successfully verified signature...
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): rekor.sigstore.dev:443
DEBUG:urllib3.connectionpool:https://rekor.sigstore.dev:443 "POST /api/v1/index/retrieve/ HTTP/1.1" 200 69
DEBUG:urllib3.connectionpool:https://rekor.sigstore.dev:443 "GET /api/v1/log/entries/9ee91f2c5444e4ff77a3a18885f46fa2b6f7e629450904d67b5920333327b90d HTTP/1.1" 200 None
DEBUG:sigstore._verify:Successfully verified Rekor entry...
OK: cosign-1.9.0.x86_64.rpm

Oh, one more important thing. The maintainers of the tool are amazing about feedback. I had some trouble initially (a few weeks ago). They sat down with me to make sure that they could understand the problem & also solved the issue I had. You can talk to the team (and other users, including me) in the slack room.

...
@blog July 2, 2022 - 00:00 • 2 months ago
New Alpha Release: Tor Browser 11.5a13 (Android, Windows, macOS, Linux)

Tor Browser 11.5a13 is now available from the Tor Browser download page and also from our distribution directory.

Tor Browser 11.5a13 updates Firefox on Windows, macOS, and Linux to 91.10.0esr.

We use the opportunity as well to update various other components of Tor Browser:

  • NoScript 11.4.6
  • Tor Launcher 0.2.36
  • Tor 0.4.7.8

This version includes important security updates to Firefox.

The full changelog since Tor Browser 11.5a12 is:

...
@blog July 2, 2022 - 00:00 • 2 months ago
New Release: Tor Browser 11.0.15 (Android, Windows, macOS, Linux)
...
@blog June 24, 2022 - 00:00 • 2 months ago
Arti 0.5.0 is released: Robustness and API improvements

Arti is our ongoing project to create a working embeddable Tor client in Rust. It’s not ready to replace the main Tor implementation in C, but we believe that it’s the future.

Right now, our focus is on making Arti production-quality, by stress-testing the code, hunting for likely bugs, and adding missing features that we know from experience that users will need. We're going to try not to break backward compatibility too much, but we'll do so when we think it's a good idea.

What's new in 0.5.0?

For a complete list of changes, have a look at the CHANGELOG.

This release adds cryptographic acceleration, a useful set of toplevel build features, reachable-address filtering, detection for failed directory downloads, and numerous cleanups.

There are also a bunch of smaller features, bugfixes, and infrastructure improvements; again, see the CHANGELOG for a more complete list.

Note that for the first time ever, we managed to go a whole release cycle without breaking source-compatibility in our arti-client APIs: for this reason, the latest version of arti-client is 0.4.1.

And what's next?

In the short term, we're working for feature-parity with Tor in netflow resistance and congestion control. We already have some backend code, but a complete implementation will take a while.

Beyond that, between now and our 1.0.0 milestone in September, we're aiming to make Arti a production-quality Tor client for direct internet access. (Onion services aren't funded yet, but we hope to change that soon.)

To do so, we need to bring Arti up to par with the C tor implementation in terms of its CPU usage, and security features. You can follow our progress on our 1.0.0 milestone.

We still plan to continue regular releases between now and then.

Here's how to try it out

We rely on users and volunteers to find problems in our software and suggest directions for its improvement. Although Arti isn't yet ready for production use, you can test it as a SOCKS proxy (if you're willing to compile from source) and as an embeddable library (if you don't mind a little API instability).

Assuming you've installed Arti (with cargo install arti, or directly from a cloned repository), you can use it to start a simple SOCKS proxy for making connections via Tor with:

$ arti proxy -p 9150

and use it more or less as you would use the C Tor implementation!

(It doesn't support onion services yet. If compilation doesn't work, make sure you have development files for libsqlite installed on your platform.)

If you want to build a program with Arti, you probably want to start with the arti-client crate. Be sure to check out the examples too.

For more information, check out the README file. (For now, it assumes that you're comfortable building Rust programs from the command line). Our CONTRIBUTING file has more information on installing development tools, and on using Arti inside of Tor Browser. (If you want to try that, please be aware that Arti doesn't support onion services yet.)

When you find bugs, please report them on our bugtracker. You can request an account or report a bug anonymously.

And if this documentation doesn't make sense, please ask questions! The questions you ask today might help improve the documentation tomorrow.

Call for feedback

Our priority for the coming months is to make Arti a production-quality Tor client, for the purposes of direct connections to the internet. (Onion services will come later.) We know some of the steps we'll need to take to get there, but not all of them: we need to know what's missing for your use-cases.

Whether you're a user or a developer, please give Arti a try, and let us know what you think. The sooner we learn what you need, the better our chances of getting it into an early milestone.

Acknowledgments

Thanks to everybody who has contributed to this release, including 0x4ndy, Alex Xu, Arturo Marquez, Dimitris Apostolou, Michael McCune, Neel Chauhan, Orhun Parmaksız, Steven Murdoch, and Trinity Pointard.

And thanks, of course, to Zcash Community Grants (formerly Zcash Open Major Grants (ZOMG)) for funding this project!

...
@kushal June 18, 2022 - 07:16 • 2 months ago
Tor 0.4.7.8 is ready

Last night I built and pushed the Tor RPM(s) for 0.4.7.8. This is a security update, so please make sure that you upgrade your relays and bridges.

You can know more about the Tor's RPM respository at https://support.torproject.org/rpm/

If you have any queries, feel free to find us over #tor channel on OFTC.

...
@anarcat June 17, 2022 - 15:34 • 2 months ago
Matrix notes

I have some concerns about Matrix (the protocol, not the movie that came out recently, although I do have concerns about that as well). I've been watching the project for a long time, and it seems more a promising alternative to many protocols like IRC, XMPP, and Signal.

This review may sound a bit negative, because it focuses on those concerns. I am the operator of an IRC network and people keep asking me to bridge it with Matrix. I have myself considered just giving up on IRC and converting to Matrix. This space is a living document exploring my research of that problem space. The TL;DR: is that no, I'm not setting up a bridge just yet, and I'm still on IRC.

This article was written over the course of the last three months, but I have been watching the Matrix project for years (my logs seem to say 2016 at least). The article is rather long. It will likely take you half an hour to read, so copy this over to your ebook reader, your tablet, or dead trees, and lean back and relax as I show you around the Matrix. Or, alternatively, just jump to a section that interest you, most likely the conclusion.

Introduction to Matrix

Matrix is an "open standard for interoperable, decentralised, real-time communication over IP. It can be used to power Instant Messaging, VoIP/WebRTC signalling, Internet of Things communication - or anywhere you need a standard HTTP API for publishing and subscribing to data whilst tracking the conversation history".

It's also (when compared with XMPP) "an eventually consistent global JSON database with an HTTP API and pubsub semantics - whilst XMPP can be thought of as a message passing protocol."

According to their FAQ, the project started in 2014, has about 20,000 servers, and millions of users. Matrix works over HTTPS but over a special port: 8448.

Security and privacy

I have some concerns about the security promises of Matrix. It's advertised as a "secure" with "E2E [end-to-end] encryption", but how does it actually work?

Data retention defaults

One of my main concerns with Matrix is data retention, which is a key part of security in a threat model where (for example) an hostile state actor wants to surveil your communications and can seize your devices.

On IRC, servers don't actually keep messages all that long: they pass them along to other servers and clients as fast as they can, only keep them in memory, and move on to the next message. There are no concerns about data retention on messages (and their metadata) other than the network layer. (I'm ignoring the issues with user registration, which is a separate, if valid, concern.) Obviously, an hostile server could log everything passing through it, but IRC federations are normally tightly controlled. So, if you trust your IRC operators, you should be fairly safe. Obviously, clients can (and often do, even if OTR is configured!) log all messages, but this is generally not the default. Irssi, for example, does not log by default. IRC bouncers are more likely to log to disk, of course, to be able to do what they do.

Compare this to Matrix: when you send a message to a Matrix homeserver, that server first stores it in its internal SQL database. Then it will transmit that message to all clients connected to that server and room, and to all other servers that have clients connected to that room. Those remote servers, in turn, will keep a copy of that message and all its metadata in their own database, by default forever. On encrypted rooms those messages are encrypted, but not their metadata.

There is a mechanism to expire entries in Synapse, but it is not enabled by default. So one should generally assume that a message sent on Matrix is never expired.

GDPR in the federation

But even if that setting was enabled by default, how do you control it? This is a fundamental problem of the federation: if any user is allowed to join a room (which is the default), those user's servers will log all content and metadata from that room. That includes private, one-on-one conversations, since those are essentially rooms as well.

In the context of the GDPR, this is really tricky: who is the responsible party (known as the "data controller") here? It's basically any yahoo who fires up a home server and joins a room.

In a federated network, one has to wonder whether GDPR enforcement is even possible at all. But in Matrix in particular, if you want to enforce your right to be forgotten in a given room, you would have to:

  1. enumerate all the users that ever joined the room while you were there
  2. discover all their home servers
  3. start a GDPR procedure against all those servers

I recognize this is a hard problem to solve while still keeping an open ecosystem. But I believe that Matrix should have much stricter defaults towards data retention than right now. Message expiry should be enforced by default, for example. (Note that there are also redaction policies that could be used to implement part of the GDPR automatically, see the privacy policy discussion below on that.)

Also keep in mind that, in the brave new peer-to-peer world that Matrix is heading towards, the boundary between server and client is likely to be fuzzier, which would make applying the GDPR even more difficult.

Update: this comment links to this post (in german) which apparently studied the question and concluded that Matrix is not GDPR-compliant.

In fact, maybe Synapse should be designed so that there's no configurable flag to turn off data retention. A bit like how most system loggers in UNIX (e.g. syslog) come with a log retention system that typically rotate logs after a few weeks or month. Historically, this was designed to keep hard drives from filling up, but it also has the added benefit of limiting the amount of personal information kept on disk in this modern day. (Arguably, syslog doesn't rotate logs on its own, but, say, Debian GNU/Linux, as an installed system, does have log retention policies well defined for installed packages, and those can be discussed. And "no expiry" is definitely a bug.

Matrix.org privacy policy

When I first looked at Matrix, five years ago, Element.io was called Riot.im and had a rather dubious privacy policy:

We currently use cookies to support our use of Google Analytics on the Website and Service. Google Analytics collects information about how you use the Website and Service.

[...]

This helps us to provide you with a good experience when you browse our Website and use our Service and also allows us to improve our Website and our Service.

When I asked Matrix people about why they were using Google Analytics, they explained this was for development purposes and they were aiming for velocity at the time, not privacy (paraphrasing here).

They also included a "free to snitch" clause:

If we are or believe that we are under a duty to disclose or share your personal data, we will do so in order to comply with any legal obligation, the instructions or requests of a governmental authority or regulator, including those outside of the UK.

Those are really broad terms, above and beyond what is typically expected legally.

Like the current retention policies, such user tracking and ... "liberal" collaboration practices with the state set a bad precedent for other home servers.

Thankfully, since the above policy was published (2017), the GDPR was "implemented" (2018) and it seems like both the Element.io privacy policy and the Matrix.org privacy policy have been somewhat improved since.

Notable points of the new privacy policies:

  • 2.3.1.1: the "federation" section actually outlines that "Federated homeservers and Matrix clients which respect the Matrix protocol are expected to honour these controls and redaction/erasure requests, but other federated homeservers are outside of the span of control of Element, and we cannot guarantee how this data will be processed"
  • 2.6: users under the age of 16 should not use the matrix.org service
  • 2.10: Upcloud, Mythic Beast, Amazon, and CloudFlare possibly have access to your data (it's nice to at least mention this in the privacy policy: many providers don't even bother admitting to this kind of delegation)
  • Element 2.2.1: mentions many more third parties (Twilio, Stripe, Quaderno, LinkedIn, Twitter, Google, Outplay, PipeDrive, HubSpot, Posthog, Sentry, and Matomo (phew!) used when you are paying Matrix.org for hosting

I'm not super happy with all the trackers they have on the Element platform, but then again you don't have to use that service. Your favorite homeserver (assuming you are not on Matrix.org) probably has their own Element deployment, hopefully without all that garbage.

Overall, this is all a huge improvement over the previous privacy policy, so hats off to the Matrix people for figuring out a reasonable policy in such a tricky context. I particularly like this bit:

We will forget your copy of your data upon your request. We will also forward your request to be forgotten onto federated homeservers. However - these homeservers are outside our span of control, so we cannot guarantee they will forget your data.

It's great they implemented those mechanisms and, after all, if there's an hostile party in there, nothing can prevent them from using screenshots to just exfiltrate your data away from the client side anyways, even with services typically seen as more secure, like Signal.

As an aside, I also appreciate that Matrix.org has a fairly decent code of conduct, based on the TODO CoC which checks all the boxes in the geekfeminism wiki.

Metadata handling

Overall, privacy protections in Matrix mostly concern message contents, not metadata. In other words, who's talking with who, when and from where is not well protected. Compared to a tool like Signal, which goes through great lengths to anonymize that data with features like private contact discovery, disappearing messages, sealed senders, and private groups, Matrix is definitely behind. (Note: there is an issue open about message lifetimes in Element since 2020, but it's not at even at the MSC stage yet.)

This is a known issue (opened in 2019) in Synapse, but this is not just an implementation issue, it's a flaw in the protocol itself. Home servers keep join/leave of all rooms, which gives clear text information about who is talking to. Synapse logs may also contain privately identifiable information that home server admins might not be aware of in the first place. Those log rotation policies are separate from the server-level retention policy, which may be confusing for a novice sysadmin.

Combine this with the federation: even if you trust your home server to do the right thing, the second you join a public room with third-party home servers, those ideas kind of get thrown out because those servers can do whatever they want with that information. Again, a problem that is hard to solve in any federation.

To be fair, IRC doesn't have a great story here either: any client knows not only who's talking to who in a room, but also typically their client IP address. Servers can (and often do) obfuscate this, but often that obfuscation is trivial to reverse. Some servers do provide "cloaks" (sometimes automatically), but that's kind of a "slap-on" solution that actually moves the problem elsewhere: now the server knows a little more about the user.

Overall, I would worry much more about a Matrix home server seizure than a IRC or Signal server seizure. Signal does get subpoenas, and they can only give out a tiny bit of information about their users: their phone number, and their registration, and last connection date. Matrix carries a lot more information in its database.

Amplification attacks on URL previews

I (still!) run an Icecast server and sometimes share links to it on IRC which, obviously, also ends up on (more than one!) Matrix home servers because some people connect to IRC using Matrix. This, in turn, means that Matrix will connect to that URL to generate a link preview.

I feel this outlines a security issue, especially because those sockets would be kept open seemingly forever. I tried to warn the Matrix security team but somehow, I don't think this issue was taken very seriously. Here's the disclosure timeline:

  • January 18: contacted Matrix security
  • January 19: response: already reported as a bug
  • January 20: response: can't reproduce
  • January 31: timeout added, considered solved
  • January 31: I respond that I believe the security issue is underestimated, ask for clearance to disclose
  • February 1: response: asking for two weeks delay after the next release (1.53.0) including another patch, presumably in two weeks' time
  • February 22: Matrix 1.53.0 released
  • April 14: I notice the release, ask for clearance again
  • April 14: response: referred to the public disclosure

There are a couple of problems here:

  1. the bug was publicly disclosed in September 2020, and not considered a security issue until I notified them, and even then, I had to insist

  2. no clear disclosure policy timeline was proposed or seems established in the project (there is a security disclosure policy but it doesn't include any predefined timeline)

  3. I wasn't informed of the disclosure

  4. the actual solution is a size limit (10MB, already implemented), a time limit (30 seconds, implemented in PR 11784), and a content type allow list (HTML, "media" or JSON, implemented in PR 11936), and I'm not sure it's adequate

  5. (pure vanity:) I did not make it to their Hall of fame

I'm not sure those solutions are adequate because they all seem to assume a single home server will pull that one URL for a little while then stop. But in a federated network, many (possibly thousands) home servers may be connected in a single room at once. If an attacker drops a link into such a room, all those servers would connect to that link all at once. This is an amplification attack: a small amount of traffic will generate a lot more traffic to a single target. It doesn't matter there are size or time limits: the amplification is what matters here.

It should also be noted that clients that generate link previews have more amplification because they are more numerous than servers. And of course, the default Matrix client (Element) does generate link previews as well.

That said, this is possibly not a problem specific to Matrix: any federated service that generates link previews may suffer from this.

I'm honestly not sure what the solution is here. Maybe moderation? Maybe link previews are just evil? All I know is there was this weird bug in my Icecast server and I tried to ring the bell about it, and it feels it was swept under the rug. Somehow I feel this is bound to blow up again in the future, even with the current mitigation.

Moderation

In Matrix like elsewhere, Moderation is a hard problem. There is a detailed moderation guide and much of this problem space is actively worked on in Matrix right now. A fundamental problem with moderating a federated space is that a user banned from a room can rejoin the room from another server. This is why spam is such a problem in Email, and why IRC networks have stopped federating ages ago (see the IRC history for that fascinating story).

The mjolnir bot

The mjolnir moderation bot is designed to help with some of those things. It can kick and ban users, redact all of a user's message (as opposed to one by one), all of this across multiple rooms. It can also subscribe to a federated block list published by matrix.org to block known abusers (users or servers). Bans are pretty flexible and can operate at the user, room, or server level.

Matrix people suggest making the bot admin of your channels, because you can't take back admin from a user once given.

The command-line tool

There's also a new command line tool designed to do things like:

  • System notify users (all users/users from a list, specific user)
  • delete sessions/devices not seen for X days
  • purge the remote media cache
  • select rooms with various criteria (external/local/empty/created by/encrypted/cleartext)
  • purge history of theses rooms
  • shutdown rooms

This tool and Mjolnir are based on the admin API built into Synapse.

Rate limiting

Synapse has pretty good built-in rate-limiting which blocks repeated login, registration, joining, or messaging attempts. It may also end up throttling servers on the federation based on those settings.

Fundamental federation problems

Because users joining a room may come from another server, room moderators are at the mercy of the registration and moderation policies of those servers. Matrix is like IRC's +R mode ("only registered users can join") by default, except that anyone can register their own homeserver, which makes this limited.

Server admins can block IP addresses and home servers, but those tools are not easily available to room admins. There is an API (m.room.server_acl in /devtools) but it is not reliable (thanks Austin Huang for the clarification).

Matrix has the concept of guest accounts, but it is not used very much, and virtually no client or homeserver supports it. This contrasts with the way IRC works: by default, anyone can join an IRC network even without authentication. Some channels require registration, but in general you are free to join and look around (until you get blocked, of course).

I have seen anecdotal evidence (CW: Twitter, nitter link) that "moderating bridges is hell", and I can imagine why. Moderation is already hard enough on one federation, when you bridge a room with another network, you inherit all the problems from that network but without the entire abuse control tools from the original network's API...

Room admins

Matrix, in particular, has the problem that room administrators (which have the power to redact messages, ban users, and promote other users) are bound to their Matrix ID which is, in turn, bound to their home servers. This implies that a home server administrators could (1) impersonate a given user and (2) use that to hijack the room. So in practice, the home server is the trust anchor for rooms, not the user themselves.

That said, if server B administrator hijack user joe on server B, they will hijack that room on that specific server. This will not (necessarily) affect users on the other servers, as servers could refuse parts of the updates or ban the compromised account (or server).

It does seem like a major flaw that room credentials are bound to Matrix identifiers, as opposed to the E2E encryption credentials. In an encrypted room even with fully verified members, a compromised or hostile home server can still take over the room by impersonating an admin. That admin (or even a newly minted user) can then send events or listen on the conversations.

This is even more frustrating when you consider that Matrix events are actually signed and therefore have some authentication attached to them, acting like some sort of Merkle tree (as it contains a link to previous events). That signature, however, is made from the homeserver PKI keys, not the client's E2E keys, which makes E2E feel like it has been "bolted on" later.

Availability

While Matrix has a strong advantage over Signal in that it's decentralized (so anyone can run their own homeserver,), I couldn't find an easy way to run a "multi-primary" setup, or even a "redundant" setup (even if with a single primary backend), short of going full-on "replicate PostgreSQL and Redis data", which is not typically for the faint of heart.

How this works in IRC

On IRC, it's quite easy to setup redundant nodes. All you need is:

  1. a new machine (with it's own public address with an open port)

  2. a shared secret (or certificate) between that machine and an existing one on the network

  3. a connect {} block on both servers

That's it: the node will join the network and people can connect to it as usual and share the same user/namespace as the rest of the network. The servers take care of synchronizing state: you do not need to worry about replicating a database server.

(Now, experienced IRC people will know there's a catch here: IRC doesn't have authentication built in, and relies on "services" which are basically bots that authenticate users (I'm simplifying, don't nitpick). If that service goes down, the network still works, but then people can't authenticate, and they can start doing nasty things like steal people's identity if they get knocked offline. But still: basic functionality still works: you can talk in rooms and with users that are on the reachable network.)

User identities

Matrix is more complicated. Each "home server" has its own identity namespace: a specific user (say @anarcat:matrix.org) is bound to that specific home server. If that server goes down, that user is completely disconnected. They could register a new account elsewhere and reconnect, but then they basically lose all their configuration: contacts, joined channels are all lost.

(Also notice how the Matrix IDs don't look like a typical user address like an email in XMPP. They at least did their homework and got the allocation for the scheme.)

Rooms

Users talk to each other in "rooms", even in one-to-one communications. (Rooms are also used for other things like "spaces", they're basically used for everything, think "everything is a file" kind of tool.) For rooms, home servers act more like IRC nodes in that they keep a local state of the chat room and synchronize it with other servers. Users can keep talking inside a room if the server that originally hosts the room goes down. Rooms can have a local, server-specific "alias" so that, say, #room:matrix.org is also visible as #room:example.com on the example.com home server. Both addresses refer to the same room underlying room.

(Finding this in the Element settings is not obvious though, because that "alias" are actually called a "local address" there. So to create such an alias (in Element), you need to go in the room settings' "General" section, "Show more" in "Local address", then add the alias name (e.g. foo), and then that room will be available on your example.com homeserver as #foo:example.com.)

So a room doesn't belong to a server, it belongs to the federation, and anyone can join the room from any serer (if the room is public, or if invited otherwise). You can create a room on server A and when a user from server B joins, the room will be replicated on server B as well. If server A fails, server B will keep relaying traffic to connected users and servers.

A room is therefore not fundamentally addressed with the above alias, instead ,it has a internal Matrix ID, which basically a random string. It has a server name attached to it, but that was made just to avoid collisions. That can get a little confusing. For example, the #fractal:gnome.org room is an alias on the gnome.org server, but the room ID is !hwiGbsdSTZIwSRfybq:matrix.org. That's because the room was created on matrix.org, but the preferred branding is gnome.org now.

As an aside, rooms, by default, live forever, even after the last user quits. There's an admin API to delete rooms and a tombstone event to redirect to another one, but neither have a GUI yet. The latter is part of MSC1501 ("Room version upgrades") which allows a room admin to close a room, with a message and a pointer to another room.

Spaces

Discovering rooms can be tricky: there is a per-server room directory, but Matrix.org people are trying to deprecate it in favor of "Spaces". Room directories were ripe for abuse: anyone can create a room, so anyone can show up in there. It's possible to restrict who can add aliases, but anyways directories were seen as too limited.

In contrast, a "Space" is basically a room that's an index of other rooms (including other spaces), so existing moderation and administration mechanism that work in rooms can (somewhat) work in spaces as well. This enables a room directory that works across federation, regardless on which server they were originally created.

New users can be added to a space or room automatically in Synapse. (Existing users can be told about the space with a server notice.) This gives admins a way to pre-populate a list of rooms on a server, which is useful to build clusters of related home servers, providing some sort of redundancy, at the room -- not user -- level.

Home servers

So while you can workaround a home server going down at the room level, there's no such thing at the home server level, for user identities. So if you want those identities to be stable in the long term, you need to think about high availability. One limitation is that the domain name (e.g. matrix.example.com) must never change in the future, as renaming home servers is not supported.

The documentation used to say you could "run a hot spare" but that has been removed. Last I heard, it was not possible to run a high-availability setup where multiple, separate locations could replace each other automatically. You can have high performance setups where the load gets distributed among workers, but those are based on a shared database (Redis and PostgreSQL) backend.

So my guess is it would be possible to create a "warm" spare server of a matrix home server with regular PostgreSQL replication, but that is not documented in the Synapse manual. This sort of setup would also not be useful to deal with networking issues or denial of service attacks, as you will not be able to spread the load over multiple network locations easily. Redis and PostgreSQL heroes are welcome to provide their multi-primary solution in the comments. In the meantime, I'll just point out this is a solution that's handled somewhat more gracefully in IRC, by having the possibility of delegating the authentication layer.

Update: this was previously undocumented, but not only can you scale the frontend workers to multiple hosts, you can also shard the backend so that tables are distributed across multiple database hots. This has been documented only on 2022-07-11, weeks after this article was written, so you will forgive me for that omission, hopefully. Obviously, this doesn't resolve the "high availability" scenario since you still have a central server for that data, but it might help resolving performance problems for very large instances.

Delegations

If you do not want to run a Matrix server yourself, it's possible to delegate the entire thing to another server. There's a server discovery API which uses the .well-known pattern (or SRV records, but that's "not recommended" and a bit confusing) to delegate that service to another server. Be warned that the server still needs to be explicitly configured for your domain. You can't just put:

{ "m.server": "matrix.org:443" }

... on https://example.com/.well-known/matrix/server and start using @you:example.com as a Matrix ID. That's because Matrix doesn't support "virtual hosting" and you'd still be connecting to rooms and people with your matrix.org identity, not example.com as you would normally expect. This is also why you cannot rename your home server.

The server discovery API is what allows servers to find each other. Clients, on the other hand, use the client-server discovery API: this is what allows a given client to find your home server when you type your Matrix ID on login.

Performance

The high availability discussion brushed over the performance of Matrix itself, but let's now dig into that.

Horizontal scalability

There were serious scalability issues of the main Matrix server, Synapse, in the past. So the Matrix team has been working hard to improve its design. Since Synapse 1.22 the home server can horizontally scale to multiple workers (see this blog post for details) which can make it easier to scale large servers.

Other implementations

There are other promising home servers implementations from a performance standpoint (dendrite, Golang, entered beta in late 2020; conduit, Rust, beta; others), but none of those are feature-complete so there's a trade-off to be made there. Synapse is also adding a lot of feature fast, so it's an open question whether the others will ever catch up. (I have heard that Dendrite might actually surpass Synapse in features within a few years, which would put Synapse in a more "LTS" situation.)

Latency

Matrix can feel slow sometimes. For example, joining the "Matrix HQ" room in Element (from matrix.debian.social) takes a few minutes and then fails. That is because the home server has to sync the entire room state when you join the room. There was promising work on this announced in the lengthy 2021 retrospective, and some of that work landed (partial sync) in the 1.53 release already. Other improvements coming include sliding sync, lazy loading over federation, and fast room joins. So that's actually something that could be fixed in the fairly short term.

But in general, communication in Matrix doesn't feel as "snappy" as on IRC or even Signal. It's hard to quantify this without instrumenting a full latency test bed (for example the tools I used in the terminal emulators latency tests), but even just typing in a web browser feels slower than typing in a xterm or Emacs for me.

Even in conversations, I "feel" people don't immediately respond as fast. In fact, this could be an interesting double-blind experiment to make: have people guess whether they are talking to a person on Matrix, XMPP, or IRC, for example. My theory would be that people could notice that Matrix users are slower, if only because of the TCP round-trip time each message has to take.

Transport

Some courageous person actually made some tests of various messaging platforms on a congested network. His evaluation was basically:

  • Briar: uses Tor, so unusable except locally
  • Matrix: "struggled to send and receive messages", joining a room takes forever as it has to sync all history, "took 20-30 seconds for my messages to be sent and another 20 seconds for further responses"
  • XMPP: "worked in real-time, full encryption, with nearly zero lag"

So that was interesting. I suspect IRC would have also fared better, but that's just a feeling.

Other improvements to the transport layer include support for websocket and the CoAP proxy work from 2019 (targeting 100bps links), but both seem stalled at the time of writing. The Matrix people have also announced the pinecone p2p overlay network which aims at solving large, internet-scale routing problems. See also this talk at FOSDEM 2022.

Usability

Onboarding and workflow

The workflow for joining a room, when you use Element web, is not great:

  1. click on a link in a web browser
  2. land on (say) https://matrix.to/#/#matrix-dev:matrix.org
  3. offers "Element", yeah that's sounds great, let's click "Continue"
  4. land on https://app.element.io/#/room%2F%23matrix-dev%3Amatrix.org and then you need to register, aaargh

As you might have guessed by now, there is a specification to solve this, but web browsers need to adopt it as well, so that's far from actually being solved. At least browsers generally know about the matrix: scheme, it's just not exactly clear what they should do with it, especially when the handler is just another web page (e.g. Element web).

In general, when compared with tools like Signal or WhatsApp, Matrix doesn't fare so well in terms of user discovery. I probably have some of my normal contacts that have a Matrix account as well, but there's really no way to know. It's kind of creepy when Signal tells you "this person is on Signal!" but it's also pretty cool that it works, and they actually implemented it pretty well.

Registration is also less obvious: in Signal, the app confirms your phone number automatically. It's friction-less and quick. In Matrix, you need to learn about home servers, pick one, register (with a password! aargh!), and then setup encryption keys (not default), etc. It's a lot more friction.

And look, I understand: giving away your phone number is a huge trade-off. I don't like it either. But it solves a real problem and makes encryption accessible to a ton more people. Matrix does have "identity servers" that can serve that purpose, but I don't feel confident sharing my phone number there. It doesn't help that the identity servers don't have private contact discovery: giving them your phone number is a more serious security compromise than with Signal.

There's a catch-22 here too: because no one feels like giving away their phone numbers, no one does, and everyone assumes that stuff doesn't work anyways. Like it or not, Signal forcing people to divulge their phone number actually gives them critical mass that means actually a lot of my relatives are on Signal and I don't have to install crap like WhatsApp to talk with them.

5 minute clients evaluation

Throughout all my tests I evaluated a handful of Matrix clients, mostly from Flathub because almost none of them are packaged in Debian.

Right now I'm using Element, the flagship client from Matrix.org, in a web browser window, with the PopUp Window extension. This makes it look almost like a native app, and opens links in my main browser window (instead of a new tab in that separate window), which is nice. But I'm tired of buying memory to feed my web browser, so this indirection has to stop. Furthermore, I'm often getting completely logged off from Element, which means re-logging in, recovering my security keys, and reconfiguring my settings. That is extremely annoying.

Coming from Irssi, Element is really "GUI-y" (pronounced "gooey"). Lots of clickety happening. To mark conversations as read, in particular, I need to click-click-click on all the tabs that have some activity. There's no "jump to latest message" or "mark all as read" functionality as far as I could tell. In Irssi the former is built-in (alt-a) and I made a custom /READ command for the latter:

/ALIAS READ script exec \$_->activity(0) for Irssi::windows

And yes, that's a Perl script in my IRC client. I am not aware of any Matrix client that does stuff like that, except maybe Weechat, if we can call it a Matrix client, or Irssi itself, now that it has a Matrix plugin (!).

As for other clients, I have looked through the Matrix Client Matrix (confusing right?) to try to figure out which one to try, and, even after selecting Linux as a filter, the chart is just too wide to figure out anything. So I tried those, kind of randomly:

  • Fractal
  • Mirage
  • Nheko
  • Quaternion

Unfortunately, I lost my notes on those, I don't actually remember which one did what. I still have a session open with Mirage, so I guess that means it's the one I preferred, but I remember they were also all very GUI-y.

Maybe I need to look at weechat-matrix or gomuks. At least Weechat is scriptable so I could continue playing the power-user. Right now my strategy with messaging (and that includes microblogging like Twitter or Mastodon) is that everything goes through my IRC client, so Weechat could actually fit well in there. Going with gomuks, on the other hand, would mean running it in parallel with Irssi or ... ditching IRC, which is a leap I'm not quite ready to take just yet.

Oh, and basically none of those clients (except Nheko and Element) support VoIP, which is still kind of a second-class citizen in Matrix. It does not support large multimedia rooms, for example: Jitsi was used for FOSDEM instead of the native videoconferencing system.

Bots

This falls a little aside the "usability" section, but I didn't know where to put this... There's a few Matrix bots out there, and you are likely going to be able to replace your existing bots with Matrix bots. It's true that IRC has a long and impressive history with lots of various bots doing various things, but given how young Matrix is, there's still a good variety:

  • maubot: generic bot with tons of usual plugins like sed, dice, karma, xkcd, echo, rss, reminder, translate, react, exec, gitlab/github webhook receivers, weather, etc
  • opsdroid: framework to implement "chat ops" in Matrix, connects with Matrix, GitHub, GitLab, Shell commands, Slack, etc
  • matrix-nio: another framework, used to build lots more bots like:
    • hemppa: generic bot with various functionality like weather, RSS feeds, calendars, cron jobs, OpenStreetmaps lookups, URL title snarfing, wolfram alpha, astronomy pic of the day, Mastodon bridge, room bridging, oh dear
    • devops: ping, curl, etc
    • podbot: play podcast episodes from AntennaPod
    • cody: Python, Ruby, Javascript REPL
    • eno: generic bot, "personal assistant"
  • mjolnir: moderation bot
  • hookshot: bridge with GitLab/GitHub
  • matrix-monitor-bot: latency monitor

One thing I haven't found an equivalent for is Debian's MeetBot. There's an archive bot but it doesn't have topics or a meeting chair, or HTML logs.

Working on Matrix

As a developer, I find Matrix kind of intimidating. The specification is huge. The official specification itself looks somewhat digestable: it's only 6 APIs so that looks, at first, kind of reasonable. But whenever you start asking complicated questions about Matrix, you quickly fall into the Matrix Spec Change specification (which, yes, is a separate specification). And there are literally hundreds of MSCs flying around. It's hard to tell what's been adopted and what hasn't, and even harder to figure out if your specific client has implemented it.

(One trendy answer to this problem is to "rewrite it in rust": Matrix are working on implementing a lot of those specifications in a matrix-rust-sdk that's designed to take the implementation details away from users.)

Just taking the latest weekly Matrix report, you find that three new MSCs proposed, just last week! There's even a graph that shows the number of MSCs is progressing steadily, at 600+ proposals total, with the majority (300+) "new". I would guess the "merged" ones are at about 150.

That's a lot of text which includes stuff like 3D worlds which, frankly, I don't think you should be working on when you have such important security and usability problems. (The internet as a whole, arguably, doesn't fare much better. RFC600 is a really obscure discussion about "INTERFACING AN ILLINOIS PLASMA TERMINAL TO THE ARPANET". Maybe that's how many MSCs will end up as well, left forgotten in the pits of history.)

And that's the thing: maybe the Matrix people have a different objective than I have. They want to connect everything to everything, and make Matrix a generic transport for all sorts of applications, including virtual reality, collaborative editors, and so on.

I just want secure, simple messaging. Possibly with good file transfers, and video calls. That it works with existing stuff is good, and it should be federated to remove the "Signal point of failure". So I'm a bit worried with the direction all those MSCs are taking, especially when you consider that clients other than Element are still struggling to keep up with basic features like end-to-end encryption or room discovery, never mind voice or spaces...

Conclusion

Overall, Matrix is somehow in the space XMPP was a few years ago. It has a ton of features, pretty good clients, and a large community. It seems to have gained some of the momentum that XMPP has lost. It may have the most potential to replace Signal if something bad would happen to it (like, I don't know, getting banned or going nuts with cryptocurrency)...

But it's really not there yet, and I don't see Matrix trying to get there either, which is a bit worrisome.

Looking back at history

I'm also worried that we are repeating the errors of the past. The history of federated services is really fascinating:. IRC, FTP, HTTP, and SMTP were all created in the early days of the internet, and are all still around (except, arguably, FTP, which was removed from major browsers recently). All of them had to face serious challenges in growing their federation.

IRC had numerous conflicts and forks, both at the technical level but also at the political level. The history of IRC is really something that anyone working on a federated system should study in detail, because they are bound to make the same mistakes if they are not familiar with it. The "short" version is:

  • 1988: Finnish researcher publishes first IRC source code
  • 1989: 40 servers worldwide, mostly universities
  • 1990: EFnet ("eris-free network") fork which blocks the "open relay", named Eris - followers of Eris form the A-net, which promptly dissolves itself, with only EFnet remaining
  • 1992: Undernet fork, which offered authentication ("services"), routing improvements and timestamp-based channel synchronisation
  • 1994: DALnet fork, from Undernet, again on a technical disagreement
  • 1995: Freenode founded
  • 1996: IRCnet forks from EFnet, following a flame war of historical proportion, splitting the network between Europe and the Americas
  • 1997: Quakenet founded
  • 1999: (XMPP founded)
  • 2001: 6 million users, OFTC founded
  • 2002: DALnet peaks at 136,000 users
  • 2003: IRC as a whole peaks at 10 million users, EFnet peaks at 141,000 users
  • 2004: (Facebook founded), Undernet peaks at 159,000 users
  • 2005: Quakenet peaks at 242,000 users, IRCnet peaks at 136,000 (Youtube founded)
  • 2006: (Twitter founded)
  • 2009: (WhatsApp, Pinterest founded)
  • 2010: (TextSecure AKA Signal, Instagram founded)
  • 2011: (Snapchat founded)
  • ~2013: Freenode peaks at ~100,000 users
  • 2016: IRCv3 standardisation effort started (TikTok founded)
  • 2021: Freenode self-destructs, Libera chat founded
  • 2022: Libera peaks at 50,000 users, OFTC peaks at 30,000 users

(The numbers were taken from the Wikipedia page and Netsplit.de. Note that I also include other networks launch in parenthesis for context.)

Pretty dramatic, don't you think? Eventually, somehow, IRC became irrelevant for most people: few people are even aware of it now. With less than a million users active, it's smaller than Mastodon, XMPP, or Matrix at this point.1 If I were to venture a guess, I'd say that infighting, lack of a standardization body, and a somewhat annoying protocol meant the network could not grow. It's also possible that the decentralised yet centralised structure of IRC networks limited their reliability and growth.

But large social media companies have also taken over the space: observe how IRC numbers peak around the time the wave of large social media companies emerge, especially Facebook (2.9B users!!) and Twitter (400M users).

Where the federated services are in history

Right now, Matrix, and Mastodon (and email!) are at the "pre-EFnet" stage: anyone can join the federation. Mastodon has started working on a global block list of fascist servers which is interesting, but it's still an open federation. Right now, Matrix is totally open, but matrix.org publishes a (federated) block list of hostile servers (#matrix-org-coc-bl:matrix.org, yes, of course it's a room).

Interestingly, Email is also in that stage, where there are block lists of spammers, and it's a race between those blockers and spammers. Large email providers, obviously, are getting closer to the EFnet stage: you could consider they only accept email from themselves or between themselves. It's getting increasingly hard to deliver mail to Outlook and Gmail for example, partly because of bias against small providers, but also because they are including more and more machine-learning tools to sort through email and those systems are, fundamentally, unknowable. It's not quite the same as splitting the federation the way EFnet did, but the effect is similar.

HTTP has somehow managed to live in a parallel universe, as it's technically still completely federated: anyone can start a web server if they have a public IP address and anyone can connect to it. The catch, of course, is how you find the darn thing. Which is how Google became one of the most powerful corporations on earth, and how they became the gatekeepers of human knowledge online.

I have only briefly mentioned XMPP here, and my XMPP fans will undoubtedly comment on that, but I think it's somewhere in the middle of all of this. It was co-opted by Facebook and Google, and both corporations have abandoned it to its fate. I remember fondly the days where I could do instant messaging with my contacts who had a Gmail account. Those days are gone, and I don't talk to anyone over Jabber anymore, unfortunately. And this is a threat that Matrix still has to face.

It's also the threat Email is currently facing. On the one hand corporations like Facebook want to completely destroy it and have mostly succeeded: many people just have an email account to register on things and talk to their friends over Instagram or (lately) TikTok (which, I know, is not Facebook, but they started that fire).

On the other hand, you have corporations like Microsoft and Google who are still using and providing email services — because, frankly, you still do need email for stuff, just like fax is still around — but they are more and more isolated in their own silo. At this point, it's only a matter of time they reach critical mass and just decide that the risk of allowing external mail coming in is not worth the cost. They'll simply flip the switch and work on an allow-list principle. Then we'll have closed the loop and email will be dead, just like IRC is "dead" now.

I wonder which path Matrix will take. Could it liberate us from these vicious cycles?

Update: this generated some discussions on lobste.rs.


  1. According to Wikipedia, there are currently about 500 distinct IRC networks operating, on about 1,000 servers, serving over 250,000 users. In contrast, Mastodon seems to be around 5 million users, Matrix.org claimed at FOSDEM 2021 to have about 28 million globally visible accounts, and Signal lays claim to over 40 million souls. XMPP claims to have "millions" of users on the xmpp.org homepage but the FAQ says they don't actually know. On the proprietary silo side of the fence, this page says

    • Facebook: 2.9 billion users
    • WhatsApp: 2B
    • Instagram: 1.4B
    • TikTok: 1B
    • Snapchat: 500M
    • Pinterest: 480M
    • Twitter: 397M

    Notable omission from that list: Youtube, with its mind-boggling 2.6 billion users...

    Those are not the kind of numbers you just "need to convince a brother or sister" to grow the network...

...
@ooni June 17, 2022 - 00:00 • 2 months ago
Measuring DoT/DoH Blocking Using OONI Probe: A Preliminary Study
When you enter a URL such as https://example.com/, under the hood, your web browser resolves the example.com domain to one or more IP addresses using the Domain Name System (DNS), a set of federated servers and protocols providing this name-to-IP-address mapping. For example, as of 2022-06-16, example.com resolves to the 93.184.216.34 (IPv4) and 2606:2800:220:1:248:1893:25c8:1946 (IPv6) addresses. Once it knows the IP addresses for the domain, the browser then uses them to fetch the requested webpage. ...
@blog June 16, 2022 - 00:00 • 2 months ago
The Tor Project 2020-2021 Annual Report

Hello Tor community,

On behalf of the team, I am thrilled to share with you the Tor Project's latest Annual Report!

In the last year, the Tor community has mobilized against increasing global censorship, more than doubled the number of bridges on the Tor network (!!), and responded to tens of thousands of user support requests. We've also worked hard to make Tor faster, improve the health of our network, and connect with our community through training and events.

One element of this year's work that inspires me, and shows the power of the Tor community, is the response to the internet censorship in Russia and Ukraine. The entire Tor community immediately jumped into action to keep people online. Seeing this passion in action, while keeping tens of thousands of Russians connected to the open internet, has been inspiring. This anonymous user shared their story about Tor, which underlines the importance of our anti-censorship efforts:

"Tor helped me a lot. Here in Russia, blocking on the Internet is extremely common... Tor helps me bypass blocking and get more privacy. For example, many wonderful websites, such as foreign services or the websites of the Russian opposition, have been blocked. I have been using Tor for many years... without it, many very important sources of useful information would be inaccessible, or accessible with great difficulty."

I hope you take a moment to read more about mobilizing against internet censorship in this year's annual report (page 4), alongside other achievements this year. Thank you for making Tor and the freedom it provides online possible.

Beyond the programatic accomplishments, I hope you also take a look at page 12, where we share the Tor Project's expenses and revenue for the 2020-2021 financial year, based on our audited 990 tax returns. We are proud to highlight that 87% of our expenses are releated to programmatic costs. That means that a significant majority of our expenses are directly releated to building Tor, improving Tor, and ensuring that Tor is accessible to everyone.

Please take a look at our Annual Report, let us know what you think about what we've shared, and while you're at it, check out the whole reports section of our website for previous year's reports!

...
@ooni June 16, 2022 - 00:00 • 2 months ago
A Quick Look at QUIC Censorship
This blog post was originally published by the Open Technology Fund to disseminate Kathrin Elmenhorst’s QUIC-and-HTTP/3 censorship research as part of her ICFP fellowship with OONI. Last year, the new network protocol QUIC was introduced. QUIC is a general-purpose transport layer network with the goal of reducing latency compared to existing protocols. Since the introduction of QUIC, we have seen rising volumes of QUIC-based web traffic in the form of HTTP/3. ...
@kushal June 15, 2022 - 10:03 • 2 months ago
Story of a space

In my case the story continued for around 2 hours. Yesterday I was trying to implement something from a given SPEC, and tried to match my output (from Rust) with the output from the Python code written by Dr. Fett.

The problem

I had to get the JSON encoding from an ordered Array (in Python it is a simple list), say ["kushal", "das"], and get the base64urlencode(sha256sum(JSON_ENCODED_STRING)) in Rust using serde_json.

Easy, isn’t?

Let us see what the encoded string looks like:

"[\"6Ij7tM-a5iVPGboS5tmvVA\",\"John\"]"

And the final checksum is otcxXDbO4_xzK0DXL_DO2INIFXIJdO1PqVM-n_gt-3g.

But, the reference implementation in Python has the checksum as fUMdn88aaoyKTHrvZd6AuLmPraGhPJ0zF5r_JhxCVZs.

It took me a few hours to notice the space after comma in the JSON encoded string from Python:

"[\"6Ij7tM-a5iVPGboS5tmvVA\", \"John\"]"

This is due to the following line from JSON spec

Insignificant whitespace is allowed before or after any of the six structural characters.

Yes, I know I should have known better, and read the original spec properly. But, in this case I learned it in the hard way.

...
@ooni June 15, 2022 - 00:00 • 2 months ago
OONI’s submission for the OHCHR report on internet shutdowns and human rights
Currently, the 50th session of the UN Human Rights Council is taking place. In response to the UN High Commissioner for Human Rights’s call for submissions in support of the OHCHR report on internet shutdowns and human rights to the 50th session of the Human Rights Council in June 2022, the OONI team provided a submission with relevant information on the occurrence of mandated disruptions of access to social media and messaging platforms over the past 5 years based on empirical OONI network measurement data. ...
@blog June 14, 2022 - 00:00 • 2 months ago
Volunteer as an alpha tester

Tor Browser receives hundreds of changes a year: from updates to Firefox – the underlying browser on which Tor Browser is based – to entirely new features designed to help protect at-risk and censored users. However, each change made to Tor Browser has the potential to introduce new and sometimes elusive bugs.

In order to find and fix these bugs before they reach the majority of our users, we apply updates to an early version of Tor Browser known as Tor Browser Alpha before releasing them more widely. Then, as a small nonprofit, we rely on a community of volunteer testers to try out our alphas before their general release in order to keep Tor Browser available on so many platforms.

Volunteering to become an alpha tester is one of the most accessible and effective ways you can help at-risk users stay connected to Tor. By spending a few minutes testing each new alpha release and reporting bugs back to our developers, you'll help provide a more stable Tor Browser for millions of users around the world.

It's also important for us to reach as diverse a group of alpha testers as possible, including those on different platforms, in different countries, who speak different languages and use Tor Browser in different ways. For example, users who are based in countries in which the Tor Network is blocked (such as Belarus, China, Russia and Turkmenistan) are invaluable in helping us test Connection Assist – a new feature designed to detect and circumvent censorship of the Tor Network automatically. Even disparities in bandwidth can be enough for one user to experience a bug while another won't.

In addition, alpha testers have the opportunity to preview new features early, like the experimental redesign of Connection Settings (formally Tor Network Settings) that we're currently testing. You can help shape the direction of major new features like this by providing feedback on their user experience as we iterate on them between each alpha release.

How to volunteer as an alpha tester

Step 1

Step 2

Step 3

Should you find a bug in Tor Browser Alpha:

  1. Check to see whether it occurs in the general release of Tor Browser. If it does, then it's an issue with Tor Browser itself, and you should follow these steps to get in touch.
  2. If this bug does not occur in the general release of Tor Browser, report it to the Tor Browser Alpha Feedback category on the Tor Forum.

As a small thank you for helping us keep Tor Browser stable and bug-free, alpha testers can collect special badges by reporting bugs on the Tor Forum too.

Alpha tester forum badges

Lastly, it's important to understand that Tor Browser Alpha is an unstable version of Tor Browser and should not be used for activities that could put you at risk. Instead, please limit your use of alpha to preview new features, test their performance and provide feedback before their release.

Happy testing!

...
@blog June 7, 2022 - 00:00 • 2 months ago
New Release: Tor Browser 11.0.14 (Android, Windows, macOS, Linux)

Tor Browser 11.0.14 is now available from the Tor Browser download page and also from our distribution directory.

This version includes important security updates to Firefox.

Tor Browser 11.0.14 updates Firefox on Windows, macOS, and Linux to 91.10.0esr.

We use the opportunity as well to update various other components of Tor Browser:

  • NoScript 11.4.6

The full changelog since Tor Browser 11.0.13 is:

  • All Platforms
  • Windows + OS X + Linux
    • Update Firefox to 91.10.0esr
  • Build System
...
@kushal May 31, 2022 - 17:08 • 3 months ago
Tor sysadmin 101 workshop for new relay operators

Tor log

On 4th June, at 19:00 UTC, we are doing an online workshop to help out new relay operators. If you ever wanted to help the Tor Project, or just curious about what is required to become a relay/bridge operator, you should join into the workshop.

The workshop is specially geared towards folks who are new to the land of Internet facing services. You will get to chat with many other operators and people from the Tor Project, and ask any doubts you have.

Register for the event, and share the news at your local groups/lists. Ask your friends to join :)

...