Presentation: Cloud Computing Interoperability

// December 1st, 2009 // 2 Comments » // amazon-ec2, cloud computing

I gave a talk about Cloud Computing Interoperability in the SoCal Code Camp at USC a week ago. I have been wanting to blog about it, but just couldn’t just because of work load.

My talk was mostly directed at the importance of interoperability in the clouds, specially in the IaaS arena. First I began with a quick definition of terms, Cloud Computing, its categories adn types. And then focused on the crust of the matter: Interoperability Efforts.

I only talked about UCI (Unified Cloud Interface) and OCCI (Open Cloud Computing Interface). I chose to talk about UCI because the effort was more ambitious and the use of  ontologies to communicate different providers attracted me a lot. UCI is a dead effort. The team provided the spec and a some-what complete Proof of Concept. But it has stalled since April, 2009.

The OCCI effort is focused mostly on IaaS, creating interoperability bridges between providers like Amazon EC2 and the like. OCCI is implemented by Open Nebula, of the RESERVOIR project, which also implements the EC2 standard (yes it is a standard!).

Anyways, I have included my presentation. I will dwell more on the matter later, when we finish our experiment.

References:

* UCI – http://groups.google.com/group/unifiedcloud
* OCCI-WG – http://www.occi-wg.org/doku.php?id=spec
* OpenNebula - http://www.opennebula.org/
* Reservoir - http://www.reservoir-fp7.eu/

Download Presentation

UPDATE: The presentation has been added to the OGF’s GridForge Wiki.

nhttp://groups.google.com/group/unifiedcloud

Post to Twitter Tweet This Post

Amazon RDS provides Relational Database Scalability

// November 3rd, 2009 // 1 Comment » // amazon-ec2, cloud computing, mysql, sql

I was very excited to hear that Amazon added a service for Relational Databases for the Cloud. It does make my life easier. One of the most important properties of the Cloud is scalability, and it is one of the most sought properties for companies with huge data flow. Using Amazon EC2 IaaS, makes it easy and flexible to scale your application, only if it is built or architected to be scalable.

There is no free lunch when it comes to scalability in the cloud. In order to take advantage of the flexibility and scalability of the Cloud, you need to architect your application in a way that it can be scaled. What Amazon provides is the infrastructure, but the application must be able to take advantage of that.

Now, this is not simple task, but it is achievable and most EC2 users are doing it now. But scaling the database is not as simple, even if you have the source code, like in the case of MySQL.

For example, let’s think about a SaaS application that is built with fine grained services which can all be scaled using Amazon’s Load Balancer. The application depends on an MySQL Server tuned for high loads sitting on an image in Amazon EC2. The services works fine most of the year, but in December, the load pumps up enough to make MySQL suffer.

Now this app is already in the cloud, so our fine grained services are scaled as needed, but the database, has become our bottleneck. What can we do? Well, there are several common options, which are not specific to the cloud and can be combined:

Now, everybody knows this works, but it comes with a cost in maintenance;  if it works in an on-premise data center it should work in the cloud. The problem with this approach, is that it doesn’t take full advantage of Cloud Scalability in Amazon EC2. The only way to take advantage of EC2’s scalability would be to actually make MySQL natively scalable, like we did with our application. Which is a huge effort, and sometimes not worth it.

Of course, Amazon reps will tell you to use SimpleDB, which I wouldn’t recommend for any mission-critical work due to the consistency issue. SimpleDB relaxes consistency for the sake of  (limited) performance, and also it doesn’t have great performance with huge datasets, which is the norm in enterprise applications. (For more information on SimpleDB’s pros and cons look here.)

Now with the new Amazon RDS, we can use a relational database that is supported by popular languages/frameworks and that can take advantage of the scalability in the cloud (specifically Amazon EC2). We just create our database using the Amazon RDS API, and point our applciation to it. If we need to scale, we can do it using the API, taking full advantage of EC2. When the data flow lowers again, we can easily release the resources using the API.

How did Amazon accomplish this? Well, it is too early to know. There are not a lot of sources of information regarding Amazon’s RDS architecture. But as it looks right now it seems like an API in front of a set of mysql databases sitting in Amazon’s EC2 images, maybe using a combination of the techniques explained before. What Amazon is aiming to provide with this new service is a database server that is easy to use, maintain and scale. Which I think is a pretty cool.

I will post more information regarding RDS after I play around with it more.

Post to Twitter Tweet This Post

Microsoft and their huge problem in the clouds…

// October 31st, 2009 // 4 Comments » // .net, cloud computing, microsoft, open source

I just recently read an article from Krishnan Subramanian, which I believe is very interesting: Microsoft’s Huge Cloud Problem.

I agree with most of the article’s comments. They have to be taken with a grain of salt, since most of is speculation. Very smart speculation, but speculation none the less. But What I do disagree completely, is the following line:

“..cloud is an evolution from the web and .NET was never a platform of choice in the web…”

I agree that the cloud is an evolution of the web, but the article talks about choice, who is it referring to? Is it the open source community? Or is the enterprise community? or is it both?

Obviously as an Open Source advocate, .Net or even Mono would not be your web platform of choice. You usually go to either PHP (which is the leader in the Open Source community) Ruby or Python (just to name a few, I know there are a lot more).

But in the enterprise world, .Net is very much relevant, and in most of the cases it is the platform of choice. I know that this is a huge market and the competition is strong, but to completely dismiss Asp.Net as not a platform of choice is far from the truth.

Asp.Net and .Net are very much relevant right now, and it will stay that way for a long time. Whether Azure succeeds or not.

It is a mistake to think that everything will be in the cloud. What will prevail are hybrid environments. That is why I think Microsoft will not only survive this (even though is going to be a really difficult climb), but it will remain relevant.

Google’s view of *EVERYTHING* in the Cloud is not very down to earth (hence the name, everything in the clouds). And in my opinion, it will never get there. A lot of things are going to be done in the cloud, and probably the majority, but not all. We are creatures of choices, and we will keep our options open.

Now with the open source movement, Microsoft has done a lot. And I actually think we should thank Miguel de Icaza and his team for this. He might be called a traitor by some, but I think he is the biggest Trojan Horse of all. He has been pushing Microsoft to open source (with the help of so many).

But let’s think about Mono for a minute. Microsoft already released the source code for .Net in a very closed license, which I see as a glass box (look but don’t touch). It is getting there, to that openness that the article is talking about. They know they have to do it. But they don’t know how.

Now, Mono is a very good example. They have been reproducing the signatures and interfaces to use .Net on Linux and it works like a charm. Also they have been adding their own mix.

Microsoft will end up releasing .Net as an Open Source project, it will not be soon though. They already have their own license for that. With what Mono has done, when Microsoft plans to release, the integration with Mono will make it easier to hit the market.

The article is right about one important thing, in order to compete in the clouds, they have to kill Windows as an Desktop OS. But I think it will prevail as Windows Azure. That is why the word “Windows” appears in there.

Just one more thing before I close this rant. I think the mistake that Netscape did with Mozilla, is a learning experience that can be applied anywhere. When Netscape decided to build their browser from Scratch instead of fixing their bloated browser at the time. They lost too much time, and they lost the browser wars. They should have fixed their browser, not start a new one, which ended up with the same problems. It eventually got fixed when the community did the right thing and fixed it with Firefox, but they didn’t not start from scratch, they fixed Mozilla.

Microsoft is the browser and we (the community) are Netscape. Are we going to kill Microsoft so that Apple or Google takes its place? And then end up with the same problems all over?

I wouldn’t really want Apple in Microsoft’s shoes. I can see what they can do with their App Store. They have so much to learn. It would be like going back to the 90’s. We already went this route with Microsoft so many times, and now, Microsoft is learning.

How about Google? I wouldn’t want Google either. They are still too young, and we haven’t seen their evil yet, which scares me a lot. They not only have a lot of power in the internet, they hold most of our data, and they want *ALL* of it. Everybody has an evil side, and Google is not any different. We just haven’t seen it yet.

Microsoft is a known evil, let’s fix it. Why change it for a new one, when this evil has already been changed so much, and it is learning to live with the community?

Well enough of rants…I’m going back to work.

Important links:

Microsoft’s Huge Cloud Problem

Post to Twitter Tweet This Post

Syntax Highlighter, WordPress & CSS

// April 26th, 2009 // No Comments » // Tools, wordpress

I recently changed my Syntax Highlighter (which was cool, but old and abandon-ware) for the SyntaxHighlighter Evolved. Which I must say I love. Old versions were really bad. They were not that flexible. But version 2.0 is VERY flexible, and looks great. So I did the change in Neonlabs and Structum.

I stumbled with several problems. The first, and the one that bothered me more, was that in the Structum page, the code seemed to be broken into weird lines.

Broken Code

This was obviously a CSS problem since I didn’t have that problem in Neonlabs. So I used several tools to see which was the offending style. I mostly looked for the ‘pre’ tag on CSS but was not successful. After looking through the whole CSS style for my theme, I found the offending line of code:

#primary code {
	display:block;
	...
}

Yep, it was a simple display attribute. I just commented that line and everything just worked.

Fixed Code

After that I tried editing the code to make it look better for the Structum page and I noticed that when it loaded it had a nasty Warning:

Warning: htmlspecialchars_decode() expects parameter 1 to be string, NULL given in /…/wp-includes/compat.php on line 105

And when I looked at the code blocks in my posts, there was nothing. They were empty.

I did a little research to see what was the problem, and it seems it is a WordPress bug. Simple variable naming bug in compat.php:

	function htmlspecialchars_decode( $str, $quote_style = ENT_COMPAT )
	{
		if ( !is_scalar( $string ) ) {
			trigger_error( 'htmlspecialchars_decode() expects parameter 1 to be string, ' . gettype( $string ) . ' given', E_USER_WARNING );
			return;
		}
	...
  	}

Can you see the problem ? The problem resides in the $string variable. It doesn’t exist, so its value is null. Changing the variable parameter for is_scalar and gettype from $string to $str fixed the problem.

This is how it should look:

	function htmlspecialchars_decode( $str, $quote_style = ENT_COMPAT )
	{
		if ( !is_scalar( $str ) ) {
			trigger_error( 'htmlspecialchars_decode() expects parameter 1 to be string, ' . gettype( $str ) . ' given', E_USER_WARNING );
			return;
		}
	...
  	}

After that, everything is working seamlessly again :D. hope this helps :D.

Post to Twitter Tweet This Post

Easy .Net Transaction Management with Transaction Scope

// April 3rd, 2009 // No Comments » // .net, SQL Server, ado.net, software development

Transactions are a common technique to ensure consistency of the data when using database applicationsfor example with Sql Server. The System.Transactions namespace in the .Net framework simplifies Transaction Managements considerably.

This time we are going to talk about TransactionScope, which is part of the System.Transactions assembly. Using TransactionScope to manage transactions is fairly simple, yet powerful. Let’s look at an simple example using ADO:

TransactionScope tranScope = new TransactionScope();

SqlConnection connection = new SqlConnection(connectionSettigs);
SqlCommand cmd = new SqlCommand(query, connection);

connection.Open();
cmd.ExecuteNonQuery();
connection.Close();

transScope.Complete();

In the example we first initialized the TransactionScope object, then we initialized the connection (very important point, we will talk about it later), execute the command, etc. And then at the end we called the method Complete for the transaction scope object.

When the transaction scope object is initialized, the transaction is effectively created and all commands after that will be protected by the transaction. After the commands are executed, then we decide to either commit the transaction or rollback. Now, for the transaction scope, the method Complete effectively commits the transaction to the database, when Dispose executes the rollback.

For some developers, is quite confusing since most developers are used to:

// Handling the transactions explicitly.
SqlConnection connection = new SqlConnection(connectionSettigs);
SqlTransaction trans = connection.BeginTransaction();

SqlCommand cmd = new SqlCommand(query, connection);
cmd.Transaction = trans;

connection.Open();
cmd.ExecuteNonQuery();
connection.Close();

connection.Commit();

Not only is the TransactionScope idiom different but also the Commit and Rollback method names are also different. The reason why the names are also different is because 1) The TransactionScope object is not a transaction per se, it is more of a transaction handler and 2) Because TransactionScope follows the Dispose pattern (just like SqlConnection). Following the Dispose pattern to implement the TransactionScope makes it easy to do this:

// Handling the transactions implicitly.
using(TransactionScope tranScope = new TransactionScope()) {

	SqlConnection connection = new SqlConnection(connectionSettigs);
	SqlCommand cmd = new SqlCommand(query, connection);

	connection.Open();
	int id = cmd.ExecuteScalar();
	connection.Close();

	// If success commit
	if(id > 0) {
		transScope.Complete();
	}
}

The example displayed above shows how to handle implicit transactions using the ‘using‘ keyword. When applying the ‘using‘ idiom on an object which implements the Dispose pattern, the ‘Dispose‘ method of such object will be called when the using block ends.

What the transaction scope is doing is registering the transaction in the connection contained in the transaction scope block. Making it easy to handle transactions in .Net code.

In the case of the example, if the transaction scope is not committed before the block ends, it is effectively rolled back automatically, since the using idiom calls the Dispose method of the selected object. This makes the use of Transactions extremely easy to read and maintain.

Now what happens if there is an exception inside of the TransactionScope block ? Well, the object’s Dispose method is called, which automatically rolls back the transaction.

Now it is very important, while using TransactionScope, that the connection is opened inside of the transaction scope block, otherwise the transaction won’t be registered in the connection.

SqlConnection connection = new SqlConnection(connectionSettigs);

// This will not work. The transaction will not be registered in the connection,
// and it will deadlock your application.
connection.Open();

// Handling the transactions implicitly.
using(TransactionScope tranScope = new TransactionScope()) {

	SqlCommand cmd = new SqlCommand(query, connection);

	int id = cmd.ExecuteScalar();

	// If success commit
	if(id > 0) {
		transScope.Complete();
	}
}

connection.Close();

The above example will not work. What happens is that when the Connection is opened inside the TransactionScope block, the transaction is registered in the connection which means that any command for that connection will be in the transaction until the connection is closed.

Be very careful to initialize the connection inside the transaction scope block, otherwise the transaction won’t register itself in the connection, and your data won’t be protected by the transaction. In some cases, any command inside of the transaction scope block, for which the connection is opened before the transaction block will result in a dead lock. The code above serves as an example of this case.

More information:
MSDN:Transaction Scope Class.

MSDN:Implementing an Implicit Transaction using Transaction Scope

Post to Twitter Tweet This Post

« Older Entries