/devlog/

DevLog #11: Gemini explosion

15/05/2020

Gemini has been featured on the front page of Hacker News a few times in the past week or two. This has let to an explosion of interest in both the protocol and the clients and servers supporting it. Naturally, Bombadillo has been among the clients people are trying out. I received a few bug reports and feature requests. The new 2.3.0 incorporates some important ones. In particular, the storing of gemini host certificate expiration datetimes. Storing these now allows Bombadillo to automatically purge an expired cert and ask the server for a new one. This had been a fairly cumbersome process that most users did not seem to know about. Initial testing shows far fewer issues dealing with certificates with this in place.

Other changes include better handling of gemini preformatted text blocks through a new config value, auto-redirection when the only difference to the URL supplied by the user is the addition of a '/' to the end for a directory, a maximum width for text rendering of gemini documents (set to 100 chars, after that wrapping will occur), eliminates an issue where confirmation of redirects was not working correctly, and makes big improvements to how gemini based relative URLs are handled. That was a big bomb of changes, lol. Most will likely go unnoticed by the average user (in a good way). I think gemini is improving all the time and that the new userbase is really driven to create a community around this. For their part Solderpunk is leading things in a good manner: clear and cautious. It is good to know that things wont just be thrown in willy-nilly, and that there is a bar of usefulness that needs to be crossed. Hopefully things will be semi-stable for a bit. In the future the big thing for Bombadillo to improve on will be client certificates. Right now there is very minimal functionality in that regard. I am hesitant about this as I have not found a way within the Go standard library to generate a self-signed certificate... I may just be missing something though. I definitely do not want to have to have users install a big library on their end just to generate certs. We'll see how things go and I'll update here when something solidifies.

I need to get bombadillo.colorfield.space set up with a gemini server soon. It would be nice for users to access information securely. I will try to get on that once this 2.3.0 release settles in. An update to the user guide on all platforms (web, gopher, gemini) probably couldnt hurt either. I hope everyone is doing well in these crazy times. I have been inside my apartment for 7 (?) weeks now only going out a small handful of times for groceries. We are doing well here though. Be safe out there!

Lastly, and a late addition to this devlog, user @dancek over on tildegit was nice enough to pitch in and help remove 'stty' as a dependency! We no longer have to call out to 'stty' but instead use actual syscalls. This should make bombadillo mroe portable and work with a wider array of system configurations. Thanks @dancek!

DevLog #10: Quick links and improved terminal state

23/04/2020

With minor release 2.2.1 we have added support for 'quick links'. The number keys on your keyboard can now be pressed, in Bombadillo, to navigate to any of the first 10 links on the current page/view (with 0 being equivalent to 10). This was a request from multiple users and should make navigating around a bit quicker and easier in certain situations (often a user will want the newest phlog post or the like, which is quite often toward the top).

In addition to the quick navigation option, this minor release sees the inclusion of a bug fix: when exiting Bombadillo on certain systems terminal line wrapping was not getting turned back on. This was due to an order of opperations issue where it as getting turned back on, but only for the alternate screen... which we would immediately leave on exit. The order has been reversed and this seems to fix the issue. If you experience any terminal state problems please reach out and let us know. In the future the plan is to switch to a better method of terminal management, but the desire is to do so without adding a dependency... so we'll see what can be made to happen (likely with the syscall module of the standard lib).

DevLog #9: Images, sort of

18/03/2020

My goodness it has been quite a time since the last Bombadillo dev log. Too long! Work has continued, though at a slightly more relaxed pace. There are a few different things that will be a part of the next release (officially 2.1.4, but I plan on releasing that into master as 2.2.0). At that time the master branch will finally have a bunch of cool features that have been developed, including:

• Stability improvements
• Adds guide for developers (DEVELOPING.md)
• Bug fixes when rendering broken gopher maps
• Updates to rendering text/gemini documents per updates to the gemini spec
• Image rendering in the browser

The last item on the list is likely the big one that users would notice. You read it right: iamge rendering in the browser (in your terminal)... sort of. Bombadillo can now render PNG, JPG, and GIF files in the browser itself. It renders the images by processing them with an error diffusion dither and then renders them with unicode braille characters. As such, it will require a terminal that can display braille characters and a font containing those characters. As with many "features" of Bomabdillo, the display of images can be turned off by entering the following in command mode:

: set showimages false

You can pass in true instead of false to turn it back on. This feature currently defaults to true in the stock install, though administrators can build Bomabdillo with a different default by updating the value in `config.go`. Below you will find a link to view some examples via asciinema.

DevLog #8: 2.1.0 released!

04/01/2020

New year, new release! Happy 2020 everyone. 2.1.0 is here. It offers more stability, some new features, a number of bug fixes, and better error handling. Check out the releases page for more details. Sorry for the short update, but we hope you enjoy the new version!

DevLog #7: 2.0.0 Release and subsequent updates

14/12/2019

I cannot believe I have not updated about the release yet. Anyone checking this log will likely know that 2.0.0 has bee released (see the releases page). I am so proud of this release and think that Bombadillo is simply the best browsing experience out there for the non-web, though I may be biased. That said, there were immediately bug reports. There have been, as of this writing, six patch udpates released to the `develop` branch of the source code repository. They range from bug fixes to new features. If you have not checked out 2.0.0 or the subsequent patch releases please check out the notes on the releases page!

The most recent new feature (found in version 2.0.6) is rendering of escape codes relating to color or text display (boldness and the like). This is accomplished in a new theme: color. If a user switches to the color theme they will not likely notice any difference from the "normal" theme... until they come accross a document using color codes. At that point they will get a nice pop of color!

Another addition since the release is the ability to set your default protocol/scheme via the new configuration option "defaultscheme". The default on most systems will be gopher, but gemini and http are also available for those users that would prefer that. Setting a default scheme allows you to enter a url without the scheme. So, "colorfield.space" instead of "gopher://colorfield.space".

DevLog #6: Make, Man, Lynx Mode

20/10/2019

Lots of good stuff has been happening! First and foremost, installation has gotten better. While the go compiler has a nice/convenient "go install" command, it does not allow for including extra files, uninstalling, etc. Users will now be able to build, install, and uninstall. Running "sudo make install" will install Bombadillo to /usr/local/bin by default. Providing an alternate path prefix is available at compile time as well. In addition to installing Bombadillo the makefile will install the man page, making documentation really easy to get to. There are also "clean" and "uninstall" targets available.

Speaking of man pages: we have a man page! It is a full featured reference of the available hot keys, commands, and configuration options. If you install via the makefile the manapge will also be installed by default.

Historically Bombadillo has, when a user opts in to the behavior, opened web links in a users default browser. This had some weaknesses though: it would not work in a terminal only environment. We have not removed this feature, but we have added notification in a terminal environment that this is not possible. Instead, we have added a new configuration option: 'lynxmode'. If you have the terminal web browser lynx installed on your system you can turn on lynxmode. This will let lynx handle the requesting and parsing of a web page. That parsed and formatted document will then be shown in Bombadillo. Allowing you to surf the web directly from Bombadillo. You will be able to follow links and input addresses. You can also download files. Things that do NOT work as a result of this arrangement: forms, most css, javascript. The lot of us feel that this fits in solidly with our mission of supporting non-web protocols. It allows us to focus on those other protocols while still allowing users to choose their internet experience. Since, lets face it, on gopher and gemini you will likely encounter some web links and it would be nice to just be able to go read the article without the hassle of opening another program. We are grateful tot he developers of lynx for providing great command line options that allow this sort of integration.

DevLog #5: Bug Fixes, Documentation

20/10/2019

Wow. It has been a long time since my last update here. The intervening time has mostly been filled with bug fixes and documentation. Here is a brief list of the things that have been accomplished:

• Using ctrl+z to background Bombadillo now works without donking up your terminal display settings
• The man page is significantly more complete
• We are switching over to a makefile for installing Bombadillo and working through getting a good workflow for the makefile
• Error messages when there is a server certificate issue is improved for Gemini
• Fixed an error in the screen drawing routine that was causing some flicker for bookmarks
• Fixed an error where filepaths were not being built correctly, which affected saving files
• Added support for viewing files on the local filesystem
• Added support for the finger protocol! This has been really fun. Try: finger://typed-hole.org for a particularly cool example

I'd like to do a more full update soon, but that should give you an idea of what has been happening. There has been a lot of conversation on the issues board of the repo about how to implement various features and fixes. I really think this release is going to be great. We are at a feature freeze point for the most part, only doing bug fixes and documentation stuff (plus the makefile). There is one more last minute feature I think I would like to add. @asdf proposed it just last night. I'll keep you all posted once we decide which way to go.

I would like to dedicate a full update to adding finger support as well. It was so simple to implement and is so much fun that I really recommend that other client designers implement it. I'll post more about it soon.

DevLog #4: Client Certificates

02/10/2019

The recent logs, and the recent work for that matter, has been for gemini support. Last log it was for trust on first use TLS support. This time it is client certificates. Figuring out which way to go with this has been a pickle. Originally I had wanted Bombadillo to generate transient certificates on each load and provide support for a host of other options and features. Well, that has been pared down quite a bit.

Two new configuration options have been added: tlscertificate and tlskey. Both options should be given an absolute path on a user's filesystem to each of those items. Bombadillo will take them and build a selfsigned certificate from them. If either points to an invalid file or is an empty string or other invalid entry then no certificate will be sent with requests. On that note, it has been set up so that the certificate never gets sent, unless a server explicitly asks for it. It does not get sent with every request. This was important to me and I felt was a big part of getting this right.

Users are responsible for generating their own cert and key files. I used openssl on my linux machine to do so. Here is an example of how I have generated my test files. If you are new to this kind of thing, this could be an ok jumping off point:

openssl req -x509 -newkey rsa:4096 -keyout key.pem -nodes -out cert.pem -days 365

This has been tested in the wild as a working solution and I hope that its simple yet flexible nature makes it usable for folks. Any time you want to use a different certificate you can just change the filepaths that are being used for certificate generation within Bombadillo, or just make them empty/junk if you do not want to be sending a client certificate at all. Both options (tlscertificate and tlskey) default to empty, so this is an opt-in feature.

That is about the long and short of it. For any that are following along with these logs and updating their Bombadillo build, the previous update's TOFU stuff has been approved and merged into the develop branch. As of this writing, the client certificate updates are awaiting some conversation among the developers before merging in... but I would expect it there in the not distant future (plus you can always pull the relevant branch, they are named appropriately).

As an aside, I have been meaning to get a gemini server set up for this site (we are up on gopher currently, just use the same address, but port 70) and just have not had time. Soon! In the meantime, I'd love to see some other servers pop up and people not involved in Gemini development get started using the protocol for whatever awesome stuff they can think up! Let me know if you spot anything new and interesting out there. New stuff seems to be popping up from time to time :)

DevLog #3: TOFU for Gemini

28/09/2019

The past few days have been spent working out a TOFU security model for Gemini. This seems to be the frontrunner for preferred security over TLS for Gemini. Having read some docs I decided to give implementation a go. For the time being I am not asking a user if they want to trust for first use, which diverges from SSH's approach. It will be a given that if the page is requested and the host has not been visited before, that the user wants to trust the site. However, some safeguards are still in place even then: the certificate offered by the host must have a hostname that matches the requested hostname, not be expired, and have started its validity window.

The basic program flow is as follows:

• User makes a request to a gemini server
• Bombadillo initiates the request and handles the TLS handshake
• Bombadillo checks to see if a certificate hash is on file for the host
• If so:
  
  • Bombadillo hashes each certificate offered by the host and compares it against the stored hash
  • If a match is found, it makes sure the hostnames match, and the dates are all valid, and then sends the content request
  • If no match is found, the user is notified of that and progress forward is not possible
  • If there is a match, but the certificate is expired, Bombadillo will attempt to add a currently valid certificate from the server and continue the transaction
• If not:
  
  • Bombadillo checks each cert for validity (host names match and the dates are all good)
  • If a valid certificate is found, it gets hashed and added to Bombadillo's certificate storage and the transaction continues
  • If no valid certificate is found, it is not possible to visit the server

As you can see from the above there are a few occasions where a user can get stuck and one occasion from which they cannot move forward. To solve the former, a command has been added to purge certificates:

:purge [hostname]

Will remove the certificate for the host provided as "[hostname]" in the above example, enabling the user to request a new one (by visiting the host again). This would generally come up if a host changed their certificate and removed a the old one. The user would be notified that the host does not have any matching certificates and they can choose whether or not they feel safe clearing out what they have and taking on something new.

The aforementioned situation in which a user is blocked completely is if the host does not offer any valid certificates at all. In this case Bombadillo considers the security risk too high and it is not possible to communicate past that point with the server. Although a user may try again later (in most cases with this there is either a problem and a user should not want to go forward or a system admin has made a minor error and it will likely get fixed in the nearterm).

The last thing to note is that if a user wants to purge all of their certificates they may enter an asterisk as the hostname:

:purge *

Like many Bombadillo commands, purge is aliased to a single character (p in this case), as a shorthand.

There is no asciinema for this DevLog, as there is not much to see visually. Maybe we'll have something of that sort with the next log.

DevLog #2: Gemini Support

19/09/2019

The move to version 2 was in part decided by gemini. I had been thinking about writing a gemini client, but after awhile it just made no sense to make something separate when I could just work it into Bombadillo (which so much work had already gone into). I think one of my favorite things so far about version 2 is the ability to follow a link from gemini to gopher (and in theory, back... if anyone uses the non-standard type: 'G', 'g' was already taken by gif). It functions seamlessly and a user will likely only know by looking at the URL.

At the time of the asciinema below, I only had response status codes 2-5 worked out. I have now added status 1 (user input) to the functional list! The only thing left is status 6... user certificates. I am dreading this one, as TLS is not a strong suit for me. I also reworked the text/gemini mimetype parsing to look and behave a lot better/cleaner. Oh! Speaking of mimetypes, the golang mailcap library I adapted from the Python library of the same name works like a champ! If you follow a link to an image file, and dont have 'terminalonly' set to true, the iamge will open in an appropriate image viewer on your system! Really neat. This feature is opt in and you can always request to just download the file like normal.

DevLog #1: Bookmarks

17/09/2019

The way the screen is drawn has been completely reworked. This should take out a lot of the flicker that had occured in v1.x. This also led to changes in how and when the bookmarks panel was drawn. I decided to update the look of it while I was at it. I think it looks much better. The asciinema linked below does not do it justice. They render in a js object of some sort and full height characters still have a little gap between them, so picture the lines actually connecting with each other instead of the dashed feel in the asciicast.

Bookmarks now fully support adding a bookmark of any protocol that Bombadillo supports (which includes the protocols which it does not natively support, but instead hires out to other programs for: http, https, telnet).

DevLog #0: v2.0.0 Early Preview

15/09/2019

Version 2.0.0 took quick shape over the course of a week or so. The changes to the drawing routine were a really big step that allowed me to remove tons of code that was meant to be modular and allow me to use it in other project... but that goal is unlikely to happen and it was really just making Bombadillo bloated. So a lot of things got cut in favor of being tailor made to the application at hand. I think it has worked out well and produced a more reliable client with ebtter features.

Here is a preview of where things were before bookmarks and gemini got worked out fully: