These days I’ve been working on tools in Go for working with data using ClickHouse as the backend.

Here’s what’s been done since April of this year:

• chutils is a package to move data in and out of ClickHouse. You can, for instance, point to a CSV, let the package determine the data types and then load a table. Or, you can run a query and output the results to a new table. You can also add new fields. Finally, you can specify legal values and what to do if the value is outside that set or is null.

• fannie is an app for loading the data made available from Fannie Mae into ClickHouse. The data is housed as a single table with the monthly values being nested arrays.

• The freddie app does the same but for the Freddie Mac data.

• toch is an app for moving data from the web or files into ClickHouse.

• fred2ch is an app for moving data from the St. Louis Fed’s Fred II database to ClickHouse.

• sampler is a package that creates a stratified sample from a source ClickHouse query. The output sample is saved to ClickHouse as is a table of the strats and counts.

• seafan is a package of model-building tools for DNN models. It has data pipelines, model-specification methods, diagnostics and more. The fitting algorithm uses the awesome gorgonia package.