* (A link to the code is available at the bottom)
While I was beginning to develop my latest personal project I was thinking about which database to use. Two years or so ago, I would have just defaulted to MySQL. It was what I was familiar with, and I’d never really had an issue with it. But then, the inevitable happened. I built apps/websites that actually had decent traffic, or ran on servers with extremely limited resources. And go figure, MySQL was a nightmare. So for this latest project, that left me with the question: Which database should I use?
Now in real life, in my opinion, the answer is “it doesn’t matter.” By this I mean, show me a company that failed solely because of the tech stack they chose. You might be able to find one or two tiny ones but in reality, it just doesn’t happen. However, as an engineer you get to deal with the concept of “Technical Debt”. Which means that you should plan to the best of your abilities, to avoid having to fight it or fix it later on down the road. But that still left me with A LOT of choices. MySQL, Postgres, MongoDB, Riak, …. and Redis?
My requirements were pretty basic. I needed the fastest possible write speeds, with reasonable select speeds. The data itself is incredibly basic. There are two types and they’re associatively related to each other. So I got to thinking about it, and I decided to create a Redis model that implemented a basic Collection design pattern. All of this works pretty well, but there were a couple challenges that I had to face.
Really there were two primary challenges. Those being: Searching (you can only search keys in Redis) and auto-incrementation (Redis has no support for this). So tackling the easiest, (and necessary) one first, auto-incrementation. This one was simple, it takes the lowercase pluralized version of the model name, and created a key with a basic integer value. The integer is increased every time a resource is created, and it’s never decreased. Making it so you could GET resource_name and use that +1 for your new resources ID – which happens automatically. The second challenge was a little more complicated. How do you search for a specific resource, if you can only search by keys? For a User model, you need at the very least to be able to search by username and password.
To accomplish this task, I decided to have a key, with an underscore delimited list of the searchable indexes. That allowed me to use KEYS with the given columns, using some kind of format like user:username_hashed-password which would return the ID of the resource. Or, user:*@gmail.com_* which would return a list of IDs. This ended up working really well. Until I realized the user could literally enter admin_username, * and it would match, since it’s all the same to Redis. Which created the need for two different methods: searchBy and searchByWildcard. The underlying code is the same, since searchBy passes the request along to searchByWildcard AFTER it strips out all uses of the character *. This is akin to using mysql_real_escape_string() in pre-PDO PHP days. Which solved that problem.
In conclusion, the Model works really well. And I can definitely see use-cases for it. However, my use-case is not one of them. I found that once there were tens of thousands of rows, iterating over the keys got unacceptably slow when performing a login since I had other searchable indexes besides just username and password. But you could take the same idea, and optimize it a lot by implementing specific searchable pairs, and then for basic datasets virtually remove the need for ever iterating over keys. So I built it, and it’s pretty cool, but I ultimately ended up switching to PostgreSQL and continuing to think about my next project which could very well incorporate my new RedisModel data accessor.
Link to the code & the Model if you want to implement it into your project: https://github.com/Ryuske/RedisModel