October 25th, 2017

Reinout van Rees: PyCon.de: wednesday lightning talks

Programing, Python, by admin.

(One of my summaries of a talk at the
2017 PyCon.de conference).

Lightning talks, so I probably won’t get all the names right. If you have
additions/corrections, mail me 🙂

Automatic screenshots of your web app – Raphael Michel

Raphael develops a web application. A web app needs
documentation. Documentation needs screenshots.

Making screenshots by hand is a lot of work. You have to do it over and over
again for every new version.

Selenium is a tool for instrumenting/steering your browser. He uses chrome
--headless
. So a proper browser, but without the GUI. py.test handles the
various testcases, including the fixtures. Django has a LiveTestServerTestcase
that is also handy.

The screenshots are written like tests. Really simple and powerfull.

Note: djangocon.eu 2018 will be in Heidelberg, Germany. 23-27
May. https://2018.djangocon.eu

Counter-intuitive optimizations – Michael Penkov

He had to make some counter-intuitive optimizations lately.

He gathers data about websites and stores the info in mongodb. 500 million
domains, more or less. He wanted, from an existing list, to know which domains
weren’t in the system yet.

Doing it in python and querying the DB was slow. In the end he exported
everything to text files and did it with linux command line tools like
sort. Much faster….

Easy releases with zest.releaser – Reinout van Rees (myself)

See http://zestreleaser.readthedocs.io/ 🙂
Easy releases of (python) programs: tagging, updating version number, new
changelog header, pypi upload, etc.

Deploying your ML applications with a single command – Anand Chitipothu

When you deploy a machine learning application, you normally want to expose
some function as an API for others to use.

They’ve written a tool called "firefly" to make this easier. https://github.com/rorodata/firefly

The big data cheat sheet – Jens Dittrich

There is no database performance problem anymore. Regular queries ought to be
1ms or 1ns: way below the human perception speed of 25ms.

Problems start when you’ve got Big Data. But: what is Big Data? How big is
big?

Big data…. there are four basic solutions to it:

  • Horizontal partitioning/sharding.
  • Replication (including caching, view.
  • Hashing (may be persisted or not).
  • Differential files/LSM. Invented five times a year. Don’t re-invent it.

So: it is mostly solved already.

See http://daimond.ai

The things they don’t tell you in university – Sophia

She recently moved from university to industry…

  • You might not have as much data as they initially told you.
  • Watch out with ‘special’ values like -999. Those might actually mean ‘None’.
  • Essential data might be completely missing.
  • You sometimes need something to break. Only you’re not allowed to break it.
  • Boss: "I don’t care if it is impossible, I’ve already sold it to two
    customers".

So: don’t trust anyone in the university if they tell you about perfect
datasets without omissions and errors 🙂

Open source days Kopenhagen

16+17 March 2018 there’ll be an open source conference in Kopenhagen. Anything
open source is fine.

https://abload.de/img/screenshot2017-10-24ajsscv.png

Photo explanation: simply a picture from my train trip (with a nice
planned detour through the Eifel) from Utrecht (NL) to Karlsruhe (DE). The

disused ironworks
in Völklingen, a UNESCO heritage site.

Back Top

Leave a Reply