mysql-proxy on ubuntu (and debian)

il corra walks you through building mysql-proxy for ubuntu.

il corra » mysql-proxy on ubuntu 7.04 feisty

First of all, there is not a packetized mysql-proxy for Ubuntu, so the only way to install it is to build it from the source

Which is great. But I’d like to take this opportunity to tell people that I’ve actually been working on packages for debian/ubuntu. They’re almost ready to be released into the wild (I’m waiting on an almost non-related event) If you’d like to play with the packaging stuff before then, check out http://launchpad.net/mysql-proxy

I’ll be sure to let everyone know when the packages themselves are in an APT repository.

Why I hate YUM

I really hate YUM, the crappy-ass apt wanna-be that all the non-Debian distros wet themselves over. Yes, it’s better than what they had before, which was NOTHING. But it is absolutely inexcusable that, when apt was already available, powerful and quick, that they wrote YUM. Not because I think everyone has to use my favorite toy, but because it’s a worse tool. It’s it’s slower, it’s stupid and it gives terrible feedback.

Let’s start with today’s beef. I tried to do this:

yum install libselinux-devel

on a just-installed Fedora 7. What I got was this:

Transaction Check Error:
file /usr/share/man/man8/matchpathcon.8.gz from install of libselinux-2.0.14-4.fc7
conflicts with file from package libselinux-2.0.13-1.fc7
file /usr/share/man/man8/selinux.8.gz from install of libselinux-2.0.14-4.fc7
conflicts with file from package libselinux-2.0.13-1.fc7

Tell anybody what the problem is? First of all, has anyone ever tried installing this package?

It turns out, what it’s trying to do is install the x86_64 and the i386 version of this package, and each one of those packages want to install those manpages. It seems fairly common to me that people might want to install this package on a 64-bit server. Why does it install the 32-bit lib? Why do the packages conflict? And why does the error message suck so bad?

Speaking of terrible error messages. Try this:

yum install kernel-dev

Wait, you might exclaim, being a RedHat person, this isn’t a Debian system, we name our packages -devel. Quite true, my bad. Except what does YUM tell me?

Loading "installonlyn" plugin
Setting up Install Process
Parsing package install arguments
Nothing to do

Hm. Does that mean the package is already installed? Is it up to date? Or do we not even have a package named kernel-dev? Here’s another way to do the same thing:

mtaylor@qualinost:~$ sudo apt-get install kernel-dev
Reading package lists... Done
Building dependency tree
Reading state information... Done
E: Couldn't find package kernel-dev

Look at that! E for error, and then it shock tells me what the problem was.

The work has already been done! If you don’t want to use APT, that’s fine, you could at least look and see what it gets right and then improve on it! I mean, that’s the way this whole Free Software thing is supposed to work, isn’t it? Granted, maybe having to specify YUM repositories in chunks of XML is an improvement, and maybe I’m not seeing all the wonderful ways in which this is better than the crappy old APT from Debian.

But I don’t think so.

All in the assumptions

So I’m not going to claim to be Kevin Closson – because I’m not. I’m also not going to wade into a shared-nothing vs. shared-storage architecture debate. And here’s why: there is no right answer.

As with anything else, it comes down to what you want to do. Look at what Kevin says in his very long-windedly (yet nicely) titled:
Nearly Free or Not, GridSQL for EnterpriseDB is Simply Better Than Real Application Clusters. It is Shared-Nothing Architecture After All! « Kevin Closson’s Oracle Blog: Platform, Storage & Clustering Topics Related to Oracle Databases

Folks, today’s applications are built on large numbers of tables and complex joins. The reason shared-nothing is nothing like RAC is because instead of only shipping functions (or tasks) and lock messages to the clustered nodes, as is the case with RAC, shared-nothing requires the shipping of data

I’m going to do my best not to make this sound like a shot – because it’s not supposed to be. But today’s applications ON ORACLE, are built on large numbers of tables and complex joins. Because that’s what you use if you have a setup that needs that sort of thing. Yeah – a shared nothing, single-contiguous image database in the 1000GB scale is going to have a whole host of issues. Likewise, I’d love to see a RAC setup try to beat MySQL Cluster in the HLR world. Good luck. There are still plenty of setups where the system is not built on large numbers of tables and complex joins. And in many of the case where they are overly massive and overly complex, the schema can be fixed.

Consider massive web property scale out where you can partition the data at the application layer. Say, an enormous social network, for instance. In a typical social network, you may have millions of users, but most of the interaction is clustered by user, so you can split a user’s data across multiple machines. It really doesn’t matter all that often to Alice what’s in Bob’s user data, except at specified points of interaction. (read – friends lists) In the places where the data does cross user boundaries, it’s in such a small subdomain of data that it can be easily split out into a smaller and simpler vertical app that does one or two smaller tasks.

In this case, you’re doing shared nothing, but you don’t have to do a join across all of the data, because of the overall structure and relation of the data. The cost-per-unit-of-storage in trying to scale to numbers that big would be ridiculous on RAC or any shared-storage solution. And part of the reason there is that you just don’t need the contiguous system. You can divide and subdivide. So the data shipping problem isn’t nearly as much of a problem.

The approach I’ve taken to the shared-nothing versus shared-disk architecture topic is one of theory versus reality. I don’t care how many people say shared-nothing is the best for one workload or the other. The point is that by measured results it is not clearly the best for one (DSS) and is certainly not fit at all for the other (OLTP).

Interesting, because I was about to make the same point in the opposite direction. First of all – no argument about shared nothing in the DSS space. It just doesn’t make a whole lot of sense, for exactly the reasons listed here. But in the OLTP space, I’d beg to differ, and I’d like to use the same theory-vs-reality moniker. The “reality” listed above is based on benchmark testing. Now I’m as big a fan as anyone of benchmarks. They’re great in some contexts. But an OLTP benchmark is theory when compared to systems out in the field doing real work. Many OLTP systems can quite easily be sharded to handle an enormous amount of data and transaction rates. Having 1000’s of parallel systems each working on their own small slice of the puzzle seems to scale in a much more linear fashion in practice than does purchasing massive RAC-like systems.

Then there is the ability to run shared-nothing on tons of smaller pieces of commodity hardware with an built in assumption that pieces of hardware are going to fail. If you want the “reality” backing that one up – see Google.

Again, I’m not trying to claim an this-vs-that victory. I’m just trying to deflect an attempt at one.

It’s not that simple. It’s not cut-and-dried. There are situations where shared nothing makes a lot of sense, and there are situations where shared strorage makes sense. Nothing is built with just a hammer – you usually also need a saw and a screwdriver … and maybe even a concrete truck.

mtstat 0.7.3

mtstat is now totally on launchpad. You can even download files.

I moved a few things around for 0.7.3. The MySQL plugins are now in mysql.mtstat instead of mtstat_mysql. (To go along with my putting the NDB/Connectors Python stuff in mysql.cluster – I’m trying to make a mysql namespace here) And I split up the mysqlqps plugin into mysqlqps, mysqlhandler and mysqlqcache. You can do multiple plugins like:


mtstat -Mmysqlqps,mysqlhandler

And you’ll get output like:

_uptime __sel__ __ins__ __del__ __upd__ _quest_|___hf__ __hnxt_ __hkey_ __rrnd_ __rnxt_

1998k      0       0       0       0       0 |     0       0       0       0       0

1998k     80       8       0       5     411 |     0     359     364       5   10144

1998k     27       8       0       6     288 |     0     240     222       1       2

1998k     86       7       0       8     531 |     0     300     411       3   10141

1998k     29       4       0       2     388 |     0      77     107       0    5062

1998k     35       1       0       2     193 |     0      24     117       1       3

1998k    100       4       0       4     704 |     0      95     356       1     665

1998k    110       7       0      24    1083 |     0     127     416       4    9470

Yoshinori pointed out a bug in the MySQLdb adapter. In the C code it passes a default port value of 3306. Of course, this means that if you don’t set the port in code but expect to pick up the port number from my.cnf, it won’t work. The patch is really, really small. The port number passed to mysql_real_connect() should be 0 – not 3306. But it might be a while before that hits. So for now, if you need to do an alternate port, you can either get a copy of MySQLdb and apply this patch:

[C] — _mysql_connections.c.orig 2007-08-03 23:20:28.000000000 -0700
+++ _mysql_connections.c 2007-08-03 23:20:33.000000000 -0700 @@ -15,7 +15,7 @@
#endif
char *host = NULL, *user = NULL, *passwd = NULL,
*db = NULL, *unix_socket = NULL;
- uint port = 3306;
+ uint port = 0;
uint client_flag = 0;
static char *kwlist[] = { “host”, “user”, “passwd”, “db”, “port”,
“unix_socket”, “conv”, [/C]

Or you can hard code your port into mysql.mtstat.mysqlbase.

Also, I’ve turned on the bug tracker on launchpad, so please let me know if you’re having any problems.