Community

Short-form technical ramblings. I'm VP of Architecture at Community.com and co-author of "Docker: Up and Running" from O'Reilly Media.

Activity

Go `sqlmock` from Data Dog can be useful for making sure you know exactly which queries are being sent to the DB. But it has a limited set of type matchers and when matchers don't match, it's so hard to figure out what was actually sent. It supports custom matchers.

I figured out a little pattern I like there I pass `t` from the test (`testing.T`) into the custom matcher's struct and then use `t.Log()` to pass back what the heck was wrong with the value passed. So many WTFs saved when tests fail. #Golang

type MicrosTimestamp struct {
    t *testing.T
}
func (r MicrosTimestamp) Match(v driver.Value) bool {
 i, ok := v.(int64)
    if !ok {
        r.t.Log(fmt.Sprintf("TESTS: MicrosTimestamp got %v which is not an int64", v))
        return false
    }
...

I'm experimenting with using a couple of #Golang database libs together that I haven't used before: https://github.com/blockloop/scan and https://github.com/Masterminds/squirrel . So far I am quite positively impressed with the simplicity and niceness of the combo. I wrote some custom code generation to generate models from the DB schema. So far I am happy.

Over the weekend I started hacking around with #TruffleRuby and it's as cool as it seems like it would be. I have done plenty of JRuby in the past and I like it quite a bit. But the debugging and profiling tools on #GraalVM are second to none. And some of the interop is definitely easier... plus the ability to use C extensions! This is the little project i'm building to test it. https://github.com/relistan/shorten-truffle . Will blog something when I've spent more time at it.

I've been writing a new Event archiving service for the new event-based environment at my new gig. This is a second chance to iterate on what we built at Community that I talked about in my blog post on the subject. This time, I'm writing in #Golang because I'm on my own on this stuff at the moment. I've taken a different approach with it by archiving events to a local copy of #DuckDB as they come off the wire. Then I use DuckDB's native Parquet and S3 support to write the events batches out to S3, where they can then be queried with Athena.

This approach seems to be good so far. I will learn more when I get it into production. I feel another blog post coming later this year...

Working on CI builds is always so painful. I prefer getting them running under Compose locally and making the remote build just do the same thing. This is better. But the turnaround times are still so long.

Leslie Lamport, of LaTeX fame, is a very accomplished mathematician and computer scientist with a Turing award for his work on “fundamental contributions to the theory and
practice of distributed and concurrent systems”. He just published a draft of his new book:

"A science of concurrent programs"

https://lamport.azurewebsites.net/tla/science.pdf

True to his pedagogic approach to everything he does, "The book assumes only that you know the math one learns before entering a university." Even the appendices are fantastic. Can only wish I'll remain this lucid at his 82 years old.

#multithreading #maths #LesleyLamport

Finally, a sensible approach to multiple #AWS accounts in #Terraform with the same configs, without wrappers or external tools...

This is nice: https://cloudcity.io/blog/2023/01/12/Handling-Multiple-Separate-Accounts-With-Terraform/

Whatever anyone says about impact on performance, running Go pprof in your production app is a huge source of wins. I have over time made many corrections to CPU and memory performance because of real data from profiling in production. These have hugely outweighed the perf impact of ongoing profiling.

one thing ktistec related that i haven't had the time for is working on build and deployment tools. there are a bunch of outstanding requests—and a few PRs—for docker builds, packaged deployments for various hosting environments, etc.

if you're interested in contributing, let me know. you only have to agree to maintain them—i won't be able to.

#ktistec

We're finally moving off of AmazonMQ with its high cost and terrible performance, onto self-hosted #RabbitMQ on Kubernetes.

Wins:
1. MUCH faster, support for quorum queues
2. Less than 1/5 the cost
3. Can run latest RabbitMQ
4. Better I/O, network
5. Can tune it however we need to for our use case

Goodbye expensive, slow, terrible AmazonMQ! Major props to team members Dan Pilch and João Britto for making it happen!