I recently attended PyCon Australia, which was my first Python conference anywhere and only my second proper tech conference, but I can state, despite the small sample size, with confidence that PyCon AU 2019 was a great conference. Below are some personal highlights from last weekend.
Day 1: Specialist Track Day
My coworkers and I bought our tickets after the T-shirt order deadline, so we didn’t get shirts, and, on the walk to the convention centre alongside the lovely Darling Harbour, a seagull dive-bombed me and got a big ole chunk of my cinnamon role for its trouble (I quickly stuffed the rest in my mouth, and have sworn biblical levels of vengeance on that seagull if ever again we should meet: I’m basically like John Wick for seagulls now). My PyCon weekend had not gotten off to an auspicious start.
Camelot and Excalibur for PDF data
Although I don’t currently have any use cases for them, I was really impressed with Vinayak Mehta’s talk on the Camelot and Excalibur packages for extracting data tables from PDF files. The way it can use spacing for tables that aren’t visually separated into cells and even has a GUI for manually separating rows and columns for tricky documents is really cool. I’ve never worked with government reports in PDF form, but I have written scripts for converting horribly-formatted government-issued spreadsheets into something resembling a pivotable table, and I have nothing but respect for the tooling (there’s a GUI and a CLI!) and flexibility (you can define character filters, column counts, and more!) that he and the other contributors have built into these packages. It almost makes me want to get data from PDFs. Almost.
Unity 3D environments for reinforcement learning
In another case of information-I-won’t-use-in-the-foreseeable-future-but-was-very-cool-nonetheless, Paris Buttfield-Addison talked about building virtual environments in Unity 3D for training reinforcement learning models that can then be used to direct robots or self-driving cars. I had heard of Unity 3D, knew it powered some of the games I play, knew that my wife worked with it once, but that was about it. So, it was interesting to see Paris quickly simulate objects and environments with realistic physical interactions by dragging and dropping a few elements in the Unity GUI tool. He then demonstrated how to combine Unity with Unity Machine Learning Agents Toolkit (for plugging TensorFlow models into Unity) by walking through the creation of a virtual racetrack, the architecture for applying reinforcement learning, the code for a reinforcement learning model, then playing the video of his bulldozer careening through his virtual, python-themed track without so much as rubbing the barrier, despite the well-known fact that such activity is racing.
I had seen the use of virtual environments to train AI for robots before at the local ML/AI meetup, but it seemed like some sort of mathy black magic at the time; the combination of Unity’s user-friendly UI and Paris’s humble speaking style made it all seem so accessible, seem that I too could one day create a self-driving car.
Threat modelling the Death Star
I know nothing about security, and I don’t find the subject particularly interesting, but my Star-Wars-fanatic wife insisted that I go to this talk. So, partly for the sake of my marriage, partly intrigued by the talk’s conceit, I went to listen to Mario Areias talk about threat modelling. Due to my current role as a basic backend dev, I’m unlikely to put into practice the techniques that I learned about, but for sheer entertainment value, Mario’s talk is worth watching. Also, if you’re interested in speaking at a conference, note how the description of the talk is written to make one want to attend even if threat modelling might not interest them personally, and how the talk itself, from slides to speech, is both informative and fun.
Day 2: Main Conference Day 1 (don’t think too hard about it)
Lessons learned building Python microservices
I’ve really been getting into DevOps and microservices over the past year, and Richard Jones’s talk was one of the two at PyCon from which I got really great information about concepts, techniques, and tools that I can implement in projects in the near future. Hearing about the challenges that he and his team have faced in stitching together microservices that use different frameworks and languages, and the design decisions that they’ve made reinforced some things that I’ve heard elsewhere and made me aware of some new possibilities. In particular, their separation of the backend into multiple layers of services (e.g. a layer that connects to the frontend and another that connects with the database) was interesting. Also, I’m really eager to try out some of the tooling that Richard mentioned. I’ve used cookiecutter a little before, but never created my own, and the idea of using it to give a consistent, framework-like directory structure to all microservices sounds like a good idea. pact.io sounds like an absolute godsend for some of the problems that I’ve been having lately with keeping track of valid request and response bodies. One solution would be to become less of a careless flake, but having a tool that enforces contracts between services sounds more reasonable than completely changing my personality.
Sufficiently advanced testing (deep-dive talk)
And this is the other talk that left me thinking, wow, I’ve got to try this stuff out, like yesterday. Zac Hatfield-Dodds’s talk, being a deep-dive, gave him time to cover some advanced testing concepts, then get into the details of the package hypothesis, which implements many of them. I was already using faker to generate random values for most of my test fixtures, but in this talk, Zac took it further by introducing me to the concept of fuzzy testing, specifically, using algorithmically-generated randomness to probe the limits of what your code can handle without breaking. In theory, this will create combinations of inputs and situations that you never thought of, hopefully uncovering edge-case bugs before they pop up in production. In my day-to-day, I work on a giant Rails monolith with many, many interconnected parts, and the number of times our imaginative users create an absolutely inconceivable menagerie of conditions that break some far-flung corner of the codebase that hasn’t been changed since that whitespace git commit from four years prior makes me really appreciate the potential value of such a technique.
Zac then presented hypothesis, of which he is a maintainer, and the ways in which we mere mortals can implement fuzzy testing in our own projects without having to understand all those fancy algorithms. I have yet to really dig into the documentation, but the package seems really intuitive, with options to limit the randomly-generated inputs within given constraints, which serves to get your tests passing, but also makes explicit your assumptions about what can and cannot happen in your app. And, as we all know, explicit is better than implicit. There also seems to be a mechanism for saving input values from failed test runs, which goes a long way toward reducing the pain of flaky tests.
When software and law are the same thing
Another example of a talk and speaker that are the whole package, Brenda Wallace’s talk was interesting, entertaining, and Brenda is a compelling speaker, combining humour and serious issues with ease. If you thought converting business requirements into code was difficult, have you ever tried to encode the logic of decades-old laws that have been changed piecemeal by this party, and that minister, over the years? Turns out different people have different ideas about what defines a ‘child’ and exactly how old four-and-a-half years is. The idea behind this project is something that had never occurred to me, and its goals (democratising understanding of laws and simplifying people’s interactions with government bureaucracy) are laudable. Having recently gained permanent residency in Australia myself (not to mention having interacted with immigration departments in four different countries), I can guarantee that even when you try to fill out all the forms correctly and go to all the right desks at all the right offices, you’re just guessing half the time, because government instructions are never so clear nor so explicit as a good old if/else.
Day 3: Main Conference Day 2
By Sunday, I was pretty exhausted. Whether it was the two full days of talks and learning and socialising, or the beers I had drunk the night before, or the whiskey I had drunk the night before, who can say? Regardless, I was pounding caffeine throughout the day, and I missed a few more talks than on previous days just to give myself a break from paying attention to stuff.
The new COBOL
For me, the principle charm of Benno Rice’s talk on COBOL and how we talk about different technologies was his survey of the history of programming languages, their roots, principle uses, and coders’ (often biased) perceptions of them. I have far more formal training in iambic pentameter than the Law of Demeter, so much of this was new information to me. Like any good talk, this one certainly inspired me to want to read more on the subject when I have some time. A simple history of the likes of COBOL, Lisp, FP, and OOP, however, would have been a little dry, so the fact that Benno was able to tie his history lesson into a broader point about how we talk about various languages and, more importantly, the people who work with them added immediate relevance to make it really engaging.
Bonus: Lightning Talks
I won’t go into specific lightning talks (though there were many good ones, some informative, some funny, some just plain weird, some all of the above), but I just wanted to say that the whole exercise was definitely a highlight of the conference for me. They were perfectly timed toward the end of Saturday and Sunday, right when the caffeine from afternoon tea was fading and the hunger for dinner was rising, to energise everyone in attendance, so that we could finish the day full of vim and vigour. When there’s only five minutes to cover one’s material, not all of the talks will go according to plan, and some speakers had to be cut off, but the consistent, quick tempo and the intellectual whiplash of hearing about so many different topics, techniques, and technologies in such a short period of time meant that even the chaos, the unexpected, contributed to my enjoyment. If you have the chance to attend a future PyCon AU, you should definitely attend the lightning talks sessions, and I highly recommend watching the videos from this year, because there is some really good stuff in there.