As mentioned in some of my previous blog entries, I'm looking for am embedded database.  At the present, I think that I will remove SQL Server CE for now.  The biggest reason is that, as part of my pet project, the application to run on Mono as well.  So, that kind of leaves SQLite and Firebird
Now, in theory, it should matter much between the two for what I want to do.  I'm planning to use ActiveRecord for my data layer, and I see that both SQLite and Firebird are supported.  So, switching between SQLite or Firebird will be pretty minimal.  What about stored procedures, I hear you say?  Well, truth be told, in my old age, I've sort of drifted away from stored procedures and look at them with a suspicious eye.  So, the fact that SQLite doesn't have stored procedures really doesn't bother or impact me at all.
I'll do a bit more comparison between the two, and then decided from there which database to use.