Entity Framework, open source and some interesting finds

As probably many people found out yesterday, Microsoft decided to truly make Entity Framework open-source, including accepting contributions from community (under certain conditions), similar to ASP.NET MVC and other frameworks that are not part of core .NET Framework.

I won’t insist on the importance of this move – it’s clearly a big step forward, a shift in mentality, a good PR move, but also a sign that EF has reached it’s maturity – probably we will not see some huge new functionality added to EF (it’s little more than an O/EM after all, not rocket science). Maybe it’s also a sign that it’s not seen anymore as a key component, that could give Microsoft some competitive advantage in the future (like Windows Phone 8, where I don’t hope to see the source code in the near future 🙂 ).

Still, this open-sourced version of EF comes with some interesting finds:

  • Microsoft seems determined to use git (instead of TFS) on CodePlex, a sign that they consider too git more appropriate for community projects
  • a quick look inside the source code shows that they are using XUnit.net for all unit tests and functional tests, instead of MSUnit (no so surprisingly, because xUnit project is coordinated by Brad Wilson, a former general manager at Microsoft). Also the original developer of NUnit now works at Microsoft and has created xUnit.net with Brad Wilson.
  • the usage of Moq comes as no surprise – there is no alternative mocking framework from MS, and is already used in other projects from Microsoft (like Enterprise Library)
  • the main branch already targets .NET 4.5 and VS2012 RC
  • the project license is Apache 2.0 (not MS-PL, but they are anyway similar)
  • some (very) early support for task-based async queries that will come with in EF6 is already there: http://entityframework.codeplex.com/SourceControl/network/forks/ishisaka/EF6/changeset/255cf51c9eab

Entity Framework and Git

Posted in .NET, Entity Framework | Tagged , , , | Leave a comment

Turda’s Gorges

Sunset light..Digging into rock..Up

Turda’s Gorges, a set on Flickr.

Puțină răcoare în iunie..

Posted in Uncategorized | Leave a comment

De ce am început să evit ASP.NET WebForms (când pot)

De ce? Unul din motive – complexitatea. În încercarea de a emula programarea RAD, component-based din WinForms și pentru web applications, WebForms a ajuns rapid un elefant căruia îi descoperi complexitatea doar când te lovești de cazuri non-triviale.

Pentru cine nu e convins, o privire pe comentariile din codul sursă de la WebForms e suficientă.
(nu contează textul în sine, ce transmit acele comments, ci mărimea lor și complexitatea la care s-a ajuns – un smell ușor de detectat)
Toate acestea într-o singura clasă de bază, “aparent” simplă – ListControl:

internal bool SaveSelectedIndicesViewState {
get {
// Must be saved when 
// 1. There is a registered event handler for SelectedIndexChanged or TextChanged.
//    For our controls, we know for sure that there is no event handler registered for 
//    SelectedIndexChanged or TextChanged so we can short-circuit that check. 
// 2. Control is not enabled or visible, because the browser's post data will not include this control
// 3. The instance is a derived instance, which might be overriding the OnSelectedIndexChanged method 
//    This is a bit hacky, since we have to cover all the four derived classes we have...
// 4. AutoPostBack is true and Adapter doesn't support JavaScript
//    For ListControls to behave the same on mobile devices
//    that simulate AutoPostBack by rendering a command button, we need to save 
//    state.
// 5. The control is paginated. 
// 6. The control contains items that are disabled.  The browser's post data will not 
//    include this data for disabled items, so we need to save those selected indices.
// 
...
...
   // always save the selectedindex
   // When we've databound, we'll have items from viewstate on the next postback. 
   // If we don't cache the selected index and reset it after we databind again, 
   // the selection goes away.  So we always have to save the selectedIndex for restore
   // after databind. 
   cachedSelectedIndex = value;
public virtual string SelectedValue { ...
  // if we're in a postback and our state is loaded or the page isn't a postback but all persistance is loaded
  // but the selection doesn't exist in the list of items, 
  // throw saying we couldn't find the selected value.
// When IDMode=Static but has no ID, what should the item IDs be? Reverting to AutoID behavior. 

Mergând un pic în clasa de bază, găsim asta:

// If the control was added after PagePreLoad, we still need to databind it because it missed its
// first change in PagePreLoad.  If this control was created by a call to a parent control's DataBind 
// in Page_Load (with is relatively common), this control will already have been databound even 
// though pagePreLoad never fired and the page isn't a postback.
if (!Page.IsPostBack) { 
    RequiresDataBinding = true;
}
// If the control was added to the page after page.PreLoad, we'll never get the event and we'll
// never databind the control.  So if we're catching up and Load happens but PreLoad never happened, 
// call DataBind.  This may make the control get databound twice if the user called DataBind on the control
// directly in Page.OnLoad, but better to bind twice than never to bind at all. 
else if (IsViewStateEnabled) { 
    RequiresDataBinding = true;
Posted in .NET, Web | Tagged , , | 3 Comments

.NET 4.5 RC – noutăți mai “obscure”

.NET Framework 4.5 urmeaza să fie lansat in curand, odată cu Visual Studio 2012 și probabil Windows 8 – toată lumea vorbește de noutătile legate de Metro, WinRT, ASP.NET WebAPI, WebSockets, async programming dar mult mai rar am găsit ceva interesant despre alte chestii noi sau modificate în noua versiune, dar mai puțin vizibile.
Din lista acestor modificari si new stuff (http://msdn.microsoft.com/en-us/library/ms171868(v=vs.110).aspx), mi s-au parut oarecum interesante:

  • suport pentru arrays mai mari de 2GB (pe sisteme 64-bit) – e interesant că cineva undeva chiar are nevoie de așa ceva (cazuri în care datele nu pot fi prelucrate pe bucăți)
  • suport pentru Unicode 6.0 (in Win8) – ce înseamnă pentru mine, e că rezultatul sortării unei colecții de stringuri poate să difere în funcție de sistemul de operare folosit, chiar dacă e aceeasi versiune de .NET
  • MEF – convention-based part registration (altfel spus, convention over configuration pentru scenarii simple de DI)
  • Suport in ASP.NET WebForms pentru unele input types din HTML5: <asp:TextBox … TextMode=”Email” … > – chestia asta e utilă mai ales la web sites optimizate pentru smartphones din noua generație, a.î. hopefully când trebuie să introduc un email sau phone number, on-screen keyboard-ul să fie optimizat pentru ceea ce vreau să introduc.
  • În sfârșit – email address validation built-in the framework – nu că ar fi greu de implementat, dar fiecare developer are propriul regexp pentru validarea unui email, așa că prefer să scriu doar:

    [EmailAddress]
    public string CustomerEmail { get; set; }
    într-o aplicație ASP.NET MVC.
  • WPF ribbon în sfârșit e inclus in framework, și se pare că au renunțat la toate acele clauze de licențiere ciudate
  • INotifyDataErrorInfo – era timpul pentru o interfață mai smart decât bătrânul IDaatErrorInfo, comună între Silverlight și WPF
  • ?singleWSDL – aș fi vrut acum câteva luni să pot genera un singur WSDL pentru un web service WCF
  • un WCF service poate fi configurat din cod mult mai ușor
  • State machine workflows se întorc în WF dupa ce au fost scoase mai demult

Din lista de mai sus lipsește Entity Framework, care va fi released cu un ritm separat față de framework, și asta e bine.

Se pare că în 4.5 au profitat de ocazie si au marcat ca obsolete sau deprecated ceea ce oricum nu mai aveau de gând să dezvolte on continuare:

  • OracleClient
  • suport pentru peer-to-peer network applications in WCF
  • passport authentication (inlocuit cu live ID) – a trecut vremea cand Microsoft zicea că Passport authentication “is ză future” 🙂
  • System.Web.Mail (era și timpul)
  • System.Web.Mobile – am avut o tentativă sa-l folosesc acum vreo 8 ani – era prea utopic pentru a avea vreun viitor, la cât de diverse erau pe vremea aia browserele pe mobile (anyone remembers WML si WAP?)
  • System.Workflow.* din .NET 3.0 – un alt exemplu de schimbare bruscă a macazului la MS, nu fără motive întemeiate
  • Microspft.VisualBasic.Compatibility – să sperăm ca până la ora actuală cine a vrut să-si porteze aplicațiile VB6, a terminat 🙂 (și cine a rămas cu aplicația pe VB6 poate a făcut o alegere bună, VB6 runtime fiind încă supported pe Windows 8, spre deosebire de .NET 1.0 )

Un poster cu what’s new am găsit la: http://www.heikniemi.net/hardcoded/wp-content/uploads/2011/10/WhatsNewNET45-en.png

O concluzie? .NET Framework a ajuns, așa cum era de așteptat, la maturitate, chestii revolutionare nu mai apar peste noapte, la cele existente se șlefuiesc colțurile, iar MS impinge puternic noul val de tehnologii (WinRT etc..).
What’s new de mai sus pot părea chestii minore, dar orice pas înainte contează pentru cine se luptă cu acea tehnologie zi de zi in aplicații reale.

Posted in .NET | Tagged , | Leave a comment

Optimistic concurrency support in HTTP and WebAPI – part 2

A short follow-up to the latest post on Optimistic concurrency control in ASP.NET WebAPI.

One question that might come up is: what should we do if, after changing a resource (entity), the client send a PUT request with the changes, but forgets to specify the original ETag value (in the ‘If-Match’ HTTP header)?
In my sample code, I just ignore this case and let the code continue, the update is performed, and the concurrency conflict is not detected – the changes done by the second user are lost (lost update case).

How can we avoid this? In the HTTO and REST ‘spirit’, we should return some error code, but which one?
Fortunately, a very recent proposal for the HTTP standard comes to rescue: RFC 6585 ‘Additional HTTP Status Codes’: http://www.rfc-editor.org/rfc/rfc6585.txt.

A new HTTP status code (428 Precondition Required) is proposed, that the request from the client must be conditional, in our case ‘If-Match’:

The 428 status code indicates that the origin server requires the request to be conditional.
Its typical use is to avoid the “lost update” problem, where a client GETs a resource’s state, modifies it, and PUTs it back to the server, when meanwhile a third party has modified the state on the server, leading to a conflict. By requiring requests to be conditional, the server can assure that clients are working with the correct copies.

How do we add support in our web service, using ASP.NET Web API?
Quite easy: in the code from my last post, just add this:

    public class ProductsController : ApiController
    {

        // PUT /api/values/5
        public void Put(int id, Product product)
        {
            // retrive the existing product from persitence
            // ...

            if (existingProduct != null)
            {
                // perform concurrency conflict check
                CheckIfProductWasModified(this.Request.Headers, existingProduct);

                //update the product
                // ...
            }
            else
            {
                throw new HttpResponseException(HttpStatusCode.NotFound);
            }
        }

        private void CheckIfProductWasModified(HttpRequestHeaders requestHeaders, Product existingProduct)
        {
            bool isConditional = false;

            if (requestHeaders.IfMatch != null)
            {
                // if the request contains an If-Match haeder with a non-empty ETag
                EntityTagHeaderValue firstHeaderVal = requestHeaders.IfMatch.FirstOrDefault();
                if ((firstHeaderVal != null) && (!string.IsNullOrEmpty(firstHeaderVal.Tag))
                    && (firstHeaderVal.Tag != "*")
                    )
                {
                    isConditional = true;
                    // compare the old and new ETag value (this can be done at DB-level using a WHERE clause)
                    // ...
                }
            }

            if (!isConditional)
            {
                throw new HttpResponseException(CreatePreconditionRequiredResponse());
            }
        }

        private static HttpResponseMessage CreatePreconditionRequiredResponse()
        {
            var resp = new HttpResponseMessage((HttpStatusCode)428)
            {

                Content = new StringContent(
@"<html>
      <head>
         <title>Precondition Required</title>
      </head>
      <body>
         <h1>Precondition Required</h1>
         <p>This request is required to be conditional;
         try using ""If-Match"".</p>
      </body>
   </html>"),
                ReasonPhrase = "Precondition Required",    
            };

            return resp;
        }


      // ...

    }

And a quick test to check if the service returns the right response in such cases:

        [TestMethod]
        public async void Put_IfEtag_Is_Missing_Returns_PreconditionRequired_Status()
        {
            // first user reads the product 11
            HttpResponseMessage getProdUser1 = await _client.GetAsync(_productUri);
            getProdUser1.EnsureSuccessStatusCode();
            Product prodUser1 = await getProdUser1.Content.ReadAsAsync<Product>();

            ///////////////
            // first user modifies (PUT) and tries to persist the same product
            prodUser1.Description = prodUser1.Description + " modif by user 1";

            var putRequest1 = GetPutRequestWithoutETagMessage(prodUser1);
            HttpResponseMessage putUser1Response = await _client.SendAsync(putRequest1);

            /////////////////////
            Assert.AreEqual(428, (int)(putUser1Response.StatusCode),
                            "The response was not precondition required!");
        }

        private HttpRequestMessage GetPutRequestWithoutETagMessage(Product product)
        {
            var requestMessage = new HttpRequestMessage(HttpMethod.Put, _productUri);
            
            // add the modified object serialized as JSON
            ObjectContent prodObjContent =
                requestMessage.CreateContent<Product>(
                product,
                MediaTypeHeaderValue.Parse(jsonMediaType),
                new MediaTypeFormatter[] { new JsonMediaTypeFormatter() },
                new FormatterSelector());

            return requestMessage;
        }

Above, we have to “force” a status code, because ASP.NET WebAPI does not know yet of 428 status code.

A natural question comes up: is there any advantage of using a specific status code in this case?
If we use it just to signal a bug in our client code, there is little value. But, if we expose a public API, a third-party client could more easily “learn” our web service and adapt to it, without having to read many pages of documentation.

Posted in .NET, C#, Web | Tagged , , , , , | 2 Comments

Optimistic concurrency control in ASP.NET WebAPI

[warning: long post 🙂 ]

Setting the stage

The first question, and an expected one would be: what has to do ASP.NET Web API with concurrency control? Until now, in most applications, this problem was approached at persistence layer level or even on database server level: when implementing optimistic concurrency control, if during an update we detect that the data was modified in the meanwhile (since it was read), probably by another user or system, usually a specific exception was thrown (or a certain error code was returned) and the client application was supposed to treat this case one way or another – usually by displaying a dialog to the user that will allow to choose a way to solve the conflict, as appropriate: by cancelling his own changes, overwriting the changes done by the other user or by merging the changes, if it’s possible and depending on the application requirements.

I won’t go into details about the “theory” of optimistic concurrency control, when it’s used and when not, how it’s implemented etc. – it’s known stuff. What is less known is that HTTP (as an application-level protocol), among many other features, comes with a built-in feature that can be used (among other things) for making the optimistic concurrency control mechanism explicit: ETags. Usually used for caching in HTTP, ETags have another possible use, suggested even by the definition from the standard: “Entity tags are used for comparing two or more entities from the same requested resource. … An entity tag MUST be unique across all versions of all entities associated with a particular resource.”

Even if the terminology is a bit different, we realize that one resource (or document) could be associated to one (or more) records from a database, the ‘entities’ can be regarded as in-memory object instances (loaded from a database entity) – all this as an approximate equivalence.

Also as an analogy, an eTag could be considered (or implemented) as a ‘version’, ‘timestamp’ associated to a database record, in general an opaque value, without a meaning for the user. We realize that the ETag, apart from it’s usage for caching, it’s an ideal candidate as a standard way to transport the values used for concurrency control. The method is not something unheard of, being suggested even by those who worked at the standard long time ago (http://www.w3.org/1999/04/Editing/) and it’s used by various document/content management systems (where it appears under the name of “unreserved checkout”).

What have all of these to do with ASP.NET WebAPI? They have, because Web API tries to provide support for HTTP-based services, where HTTP is made explicit as an application-level protocol, instead of being hidden/abstracted away.

Back on earth..
Enough theory for now. How can be an ETag used in a practical way to offer support for optimistic concurrency control, at HTTP level?
Some pseudocode:

  • a client application makes a request for a certain resource: GET /products/23
  • if the resource does exists, the server will send back a response that will contain, apart from the resource itself, the ETag value for that resource, as a field in the HTTP header (ex.: ETag: “s2hk707Mvk+GqzxNe+lbOQ==”)
  • the client application displays the resource, the user (1) modifies it (on the client)
  • during this time, another user (2) (or another system) can edit and modify the resource without problems. On the server, the ETag is updated when the resource is persisted
  • the first user (1) finishes it’s changes and press a ‘Save’ button
  • the client application sends a PUT request to the server, with the modified resource and the original ETag, as a field in the HTTP header, like:
    If-Match: “<original ETag value>”
  • on the server, if we can verify (one way or another) that the resource was modified in the meanwhile (usually by comparing the original and current etags), the update won’t be performed and a response will be returned with 412 (Precondition Failed) code, where ‘precondition’ is the “If-Match” condition from the change request (some services return 409 Conflict, so be prepared fro that)
  • it’s up to the client application to decide what to do with this response, like: display to the user a message that let him to choose between different ways to solve the conflict (“owerwrite the changes done by other users”, “cancel my changes” etc.), optionally by showing a list of changes done by the other user

How does ASP.NET WebAPI helps us to implement something like this?
Easy: the actions from an ApiController offer an easy access to the HTTP headers – the rest is up to us. I let the code speak by itself 🙂

(if you read this a few years from now: the code is tested using .NET Framework 4.5 beta – it might be changed until the final release, as it happened in the past)

    public class ProductsController : ApiController
    {
        private static IList _products = new List(); // dummy product repository

        static ProductsController()
        {
            // dummy data
            _products.Add(new Product() { Id = 10, Code = "P1", Description = "Product 1", Price = 123.45m,
                                          Version = Guid.NewGuid().ToByteArray() });
            _products.Add(new Product() { Id = 11, Code = "P2", Description = "Product 2", Price = 567.47m,
                                          Version = Guid.NewGuid().ToByteArray() });
            _products.Add(new Product() { Id = 12, Code = "P3", Description = "Product 3", Price = 100.22m,
                                          Version = Guid.NewGuid().ToByteArray() });
        }

        // GET /api/products
        public IEnumerable Get()
        {
            return _products;
        }

        // GET /api/products/11
        public HttpResponseMessage Get(int id)
        {
            Product prod = (from p in _products
                         where p.Id == id
                         select p).FirstOrDefault();
            if (prod == null)
            {
                throw new HttpResponseException(HttpStatusCode.NotFound);
            }
            else
            {
                var response = new HttpResponseMessage(prod, HttpStatusCode.OK);
                response.Headers.ETag = new System.Net.Http.Headers.EntityTagHeaderValue("\""
                    + Convert.ToBase64String(prod.Version, Base64FormattingOptions.None) + "\"");
                return response;
            }
        }

        // PUT /api/values/5
        public void Put(int id, Product product)
        {
            // retrive the existing product from persitence
            Product existingProduct = (from p in _products
                            where p.Id == id
                            select p).FirstOrDefault();

            if (existingProduct != null)
            {
                // perform concurrency conflict check
                CheckIfProductWasModified(this.Request.Headers, existingProduct);

                //update the product
                existingProduct.Code = product.Code;
                existingProduct.Description = product.Description;
                existingProduct.Price = product.Price;
                // this should be done by the persistence layer (DB, etc..)
                existingProduct.Version = Guid.NewGuid().ToByteArray();

            }
            else
            {
                throw new HttpResponseException(HttpStatusCode.NotFound);
            }
        }

        private void CheckIfProductWasModified(HttpRequestHeaders requestHeaders, Product existingProduct)
        {
            if (requestHeaders.IfMatch != null)
            {
                // if the request contains an If-Match haeder with a non-empty ETag
                EntityTagHeaderValue firstHeaderVal = requestHeaders.IfMatch.FirstOrDefault();
                if ((firstHeaderVal != null) && (!string.IsNullOrEmpty(firstHeaderVal.Tag))
                    && (firstHeaderVal.Tag != "*")
                    )
                {
                    // compare the old and new ETag value (this can be done at DB-level using a WHERE clause)
                    string encodedNewTagValue = firstHeaderVal.Tag.Trim("\"".ToCharArray());
                    string encodeExistingTagValue = Convert.ToBase64String(existingProduct.Version, Base64FormattingOptions.None);
                    if (!encodedNewTagValue.Equals(encodeExistingTagValue, StringComparison.Ordinal))
                    {
                        // concurrency conflict if the resource was modified
                        throw new HttpResponseException(HttpStatusCode.PreconditionFailed);
                    }
                }
            }
        }

        // POST /api/products
        public HttpResponseMessage Post(Product product)
        {
            _products.Add(product);
            var response = new HttpResponseMessage(product, HttpStatusCode.Created);
            response.Headers.Location = new Uri(Request.RequestUri,
                "/api/products/" + product.Id.ToString(CultureInfo.InvariantCulture));
            return response;
        }

    }

The highlighted lines show the relevant parts:
– at GET, we set the ETag for the returned resource, by using HttpRespnseMessage.Headers.ETag (the value must be enclosed in double quotes (“)
GET with ETag
– at PUT (modify), if this.Request.Headers.IfMatch has a value (that it’s not “*”), we compare it with the ETag of the persisted resource (in DB or cache)
– if they don’t match, we return a response with HttpStatusCode.PreconditionFailed
PUT with ETag

Finally, some tests that show what we expect to happen:

    [TestClass]
    public class ProductsControllerIntegrationTests
    {
        private HttpClient _client;
        private string _productUri = "http://ipv4.fiddler/TestETag/api/products/11";
        private readonly string jsonMediaType = "application/json";

        [TestInitialize]
        public void Init()
        {
            _client = new HttpClient();
        }

        [TestMethod]
        public async Task Get_ReturnsTheETagInTheHeaders()
        {
            HttpResponseMessage httpResp = await _client.GetAsync(_productUri);

            Assert.IsNotNull(httpResp.Headers.ETag, "No ETag was included in the response headers!");
            Assert.IsTrue(!string.IsNullOrEmpty(httpResp.Headers.ETag.Tag), "An empty ETag was received!");

        }

        [TestMethod]
        public async Task Put_IfTheProductWasModified_ReturnsPreconditionFailed()
        {
            // first user reads the product 11
            HttpResponseMessage getProdUser1 = await _client.GetAsync(_productUri);
            getProdUser1.EnsureSuccessStatusCode();
            Product prodUser1 = await getProdUser1.Content.ReadAsAsync();

            ///////////////
            // second user reads the product 11
            HttpResponseMessage getProdUser2 = await _client.GetAsync(_productUri);
            getProdUser2.EnsureSuccessStatusCode();
            Product prodUser2 = await getProdUser2.Content.ReadAsAsync();

            //second users modifies (PUT) and persist the product
            prodUser2.Description = prodUser2.Description + " modif by user 2";

            var putRequest2 = GetPutRequestMessage(prodUser2);
            // we have to use SendAsync in order to be able to set any header, including If-Match
            HttpResponseMessage putUser2Response = await _client.SendAsync(putRequest2);
            putUser2Response.EnsureSuccessStatusCode();

            ///////////////
            // first user modifies (PUT) and tries to persist the same product
            prodUser1.Description = prodUser2.Description + " modif by user 1";

            var putRequest1 = GetPutRequestMessage(prodUser1);
            HttpResponseMessage putUser1Response = await _client.SendAsync(putRequest1);

            /////////////////////
            Assert.AreEqual(HttpStatusCode.PreconditionFailed, putUser1Response.StatusCode,
                            "The response was not precondition failed!");

        }

        private HttpRequestMessage GetPutRequestMessage(Product product)
        {
            var requestMessage = new HttpRequestMessage(HttpMethod.Put, _productUri);
            // add an 'If-Match' header
            requestMessage.Headers.IfMatch.ParseAdd("\"" + Convert.ToBase64String(product.Version) + "\"");
            // add the modified object serialized as JSON
            ObjectContent prodObjContent =
                requestMessage.CreateContent(
                product,
                MediaTypeHeaderValue.Parse(jsonMediaType),
                new MediaTypeFormatter[] { new JsonMediaTypeFormatter() },
                new FormatterSelector());

            return requestMessage;
        }

        [TestCleanup]
        public void Cleanup()
        {
            if (_client != null)
            {
                _client.Dispose();
            }
        }

    }

Obviously, it doesn’t matter the language or framework used on the client – I used HttpClient from ASP.NET Web API because it was at hand, but I could use JavaScript, C++ or something else.

For a real application, of course, there are a few more steps to be done: a lot of refactoring for having some reusable code, extracting the code dealing with concurrency checks in a separate class – separation of concerns (probably using a DelagatingChanell, like it’s described at http://javiercrespoalvez.com/2011/06/etags-and-optimistic-concurrency.html or http://codebetter.com/howarddierking/2011/07/01/automatic-etag-management-with-web-api-message-handlers/), using a real persistence solution (like a database). Obviously, the above tests are only some ‘integration’ tests – some real unit tests would be useful.

What is not relevant in the above code:
– the format used for ETag – I used byte[] only because it’s easier to map to rowversion columns from MS SQL Server, when using EF, but equally good would be an int or GUID as long as make sure it’s unique
– the way in which the ETag is encodded in the HTTP header: I used Convert.ToBase64String only because it’s a convenient and safe way to encode an array of bytes (in both directions)
– the format used to serialize the resource (entity) – it can be JSON as above, but also XML or something else

What is not discussed in this post: the role played by ETag in HTTP caching, that is important, if used.

A bit of context:
In those cases when our REST-like service is simple enough to be exposed using the OData protocol proposed by Microsoft, the protocol will use ETags for concurrency support: http://www.odata.org/documentation/operations#ConcurrencycontrolandETags
Also the Microsoft framework that uses the OData protocol, like WCF Data Services (former ADO.NET Data Services, codename Astoria) is using HTTP Etags for concurrency, so when we can use that, it’s already baked for us (http://msdn.microsoft.com/en-us/data/hh127792 ; http://blogs.msdn.com/b/astoriateam/archive/2008/04/22/optimistic-concurrency-data-services.aspx).
Even if it might not be obvious, the OData protocol (based on HTTP) can be found in many places: Sharepoint 2010 services, Excel services, Azure Storage.

Outside Microsoft world, Raven DB API also is using ETags for concurrency: http://ravendb.net/docs/http-api/http-api-comcurrency, and some GData (Google Data) services do the same: https://developers.google.com/gdata/docs/2.0/reference#ResourceVersioning

Later edit: a short follow up: part 2

Posted in .NET, Web | Tagged , , , | 18 Comments

Optimistic concurrency control în ASP.NET WebAPI

[warning: loong post 🙂 ]
Prima întrebare, și oarecum de așteptat ar fi: ce treabă are ASP.NET Web API cu concurrency control? Până acuma, în majoritatea aplicațiilor, problema asta se trata la nivelul layerului de persistență sau chiar la nivelul database server-ului: în cazul optimistic concurrency control, dacă la update se detecta că datele au fost modificate între timp (comparativ cu momentul citirii lor), cel mai probabil de către alt user sau sistem, cel mai adesea se arunca o exceptie specifică (sau se returna un cod de eroare specific) și aplicația client trebuia sa o trateze într-un fel sau altul – de obicei afișand un dialog utilizatorului și permițăndu-i sa aleagă o metodă de rezolvare a conflictului, dupa caz: cancel la modificările proprii, suprascrierea modificarilor facute de celălalt user sau merge la modificări, dacă e posibil și în funție de cerințele aplicației.
Nu voi intra în detalii legate de “teoria” optimistic concurrency control, cănd se folosește și când nu, cum se implementează etc. – e o poveste cunoscută.

Ceea ce e mai puțin cunoscut e că HTTP-ul (ca application-level protocol), printre multe alte features, vine cu un feature built-in ce poate fi folosit, (și) pentru a face explicit mecanismul de optimistic concurrency control: ETags.Folosit de obicei pentru caching in HTTP, ETag-urile mai au o posibilă utilizare, sugerată chiar de definiția din standard: “Entity tags are used for comparing two or more entities from the same requested resource. … An entity tag MUST be unique across all versions of all entities associated with a particular resource.”

Chiar dacă terminologia e puțin diferită, ne dăm sema că unei resurse (sau document) îi corespunde un record (sau mai multe) din database, ‘entities’ le putem privi ca fiind instanțe de obiecte în memorie (reprezentări concrete în memorie ale unei entități din database), asta ca o echivalență aproximativă.

Tot ca analogie, eTag-ul poate fi echivalat (sau implementat) ca un “version”, “timestamp” asociat unui database record, în general o valoare “opacă”, fară o semnificație anume pentru user.Ne dăm astfel seama ca, ETag-ul, pe langă folosirea lui pentru caching, e un candidat ideal pentru a asigura o metodă standard de a transporta valorile folosite pentru concurrency control. Metoda nu e ceva nou și nemaivăzut, fiind sugerată chiar de cei ce au lucrat la standard cu multă vreme în urmă (http://www.w3.org/1999/04/Editing/) și folosită de diverse sisteme de document/content management (unde apare sub numele de “unreserved checkout”).Ce legătură au toate astea cu ASP.NET WebAPI? Au, deoarece Web API încearcă să ofere suport pentru servicii bazate pe HTTP ca application protocol și care fac HTTP-ul explicit și îi expun avantajele, în loc să îl ascundă/abstractizeze.

Dar destul cu teoria. Cum poate fi folosit la modul concret un ETag pentru a asigura suport pentru optimistic concurrency control, la nivel de HTTP?
Pseudocod:

  • o aplicație client face un request pentru o anumită resursă: GET /products/23
  • dacă resursa există, server-ul va trimite un răspuns ce va conține, pe lânga resursa propriu-zisă, valoarea ETag-ului asociat resursei, ca și field în header-ul HTTP (ex.: ETag: “s2hk707Mvk+GqzxNe+lbOQ==”)
  • aplicația client afișează resursa, user-ul (1) o modifică (pe client)
  • în acest timp un alt user (2) (sau sistem) editează și modifică aceeași resursă fără probleme. Pe server, ETag-ul este updatat în momentul cand resursa e persistată
  • primul user (1) termină de facut modificările și apasă butonul “Save”
  • aplicația client trimite un request de tip PUT la server, cu resursa modificată și cu ETag-ul original, sub forma unui field in header-ul HTTP, de forma:
    If-Match: “<original ETag value>”
  • pe server, dacă se verifică (intr-un fel sau altul) ca resursa există dar a fost modificată între timp (cel mai adesea comparând etag-ul original cu cel curent), update-ul nu va fi efectuat și se va intoarce un response cu codul 412 (Precondition Failed) unde “precondition” e chiar condiția exprimată de “If-Match” în requestul de modificare
  • rămâne la latitudinea aplicației client ce face cu acest răspuns: afișează un mesaj la user prin care îl lasă să aleagă intre diverse variante de soluționare a conflictului (“owerwrite the changes done by other users”, “cancel my changes” etc.), eventual afișând și o listă cu modificările facute de celălalt user

Cum ne ajută ASP.NET WebApi să implementăm așa ceva?
Simplu: actiunile dintr-un ApiController oferă un access facil la headerele HTTP – restul ramane în responsabilitatea noastră. Las codul să vorbească 🙂

    public class ProductsController : ApiController
    {
        private static IList _products = new List(); // dummy product repository

        static ProductsController()
        {
            // dummy data
            _products.Add(new Product() { Id = 10, Code = "P1", Description = "Product 1", Price = 123.45m,
                                          Version = Guid.NewGuid().ToByteArray() });
            _products.Add(new Product() { Id = 11, Code = "P2", Description = "Product 2", Price = 567.47m,
                                          Version = Guid.NewGuid().ToByteArray() });
            _products.Add(new Product() { Id = 12, Code = "P3", Description = "Product 3", Price = 100.22m,
                                          Version = Guid.NewGuid().ToByteArray() });
        }

        // GET /api/products
        public IEnumerable Get()
        {
            return _products;
        }

        // GET /api/products/11
        public HttpResponseMessage Get(int id)
        {
            Product prod = (from p in _products
                         where p.Id == id
                         select p).FirstOrDefault();
            if (prod == null)
            {
                throw new HttpResponseException(HttpStatusCode.NotFound);
            }
            else
            {
                var response = new HttpResponseMessage(prod, HttpStatusCode.OK);
                response.Headers.ETag = new System.Net.Http.Headers.EntityTagHeaderValue("\""
                    + Convert.ToBase64String(prod.Version, Base64FormattingOptions.None) + "\"");
                return response;
            }
        }

        // PUT /api/values/5
        public void Put(int id, Product product)
        {
            // retrive the existing product from persitence
            Product existingProduct = (from p in _products
                            where p.Id == id
                            select p).FirstOrDefault();

            if (existingProduct != null)
            {
                // perform concurrency conflict check
                CheckIfProductWasModified(this.Request.Headers, existingProduct);

                //update the product
                existingProduct.Code = product.Code;
                existingProduct.Description = product.Description;
                existingProduct.Price = product.Price;
                // this should be done by the persistence layer (DB, etc..)
                existingProduct.Version = Guid.NewGuid().ToByteArray();

            }
            else
            {
                throw new HttpResponseException(HttpStatusCode.NotFound);
            }
        }

        private void CheckIfProductWasModified(HttpRequestHeaders requestHeaders, Product existingProduct)
        {
            if (requestHeaders.IfMatch != null)
            {
                // if the request contains an If-Match haeder with a non-empty ETag
                EntityTagHeaderValue firstHeaderVal = requestHeaders.IfMatch.FirstOrDefault();
                if ((firstHeaderVal != null) && (!string.IsNullOrEmpty(firstHeaderVal.Tag))
                    && (firstHeaderVal.Tag != "*")
                    )
                {
                    // compare the old and new ETag value (this can be done at DB-level using a WHERE clause)
                    string encodedNewTagValue = firstHeaderVal.Tag.Trim("\"".ToCharArray());
                    string encodeExistingTagValue = Convert.ToBase64String(existingProduct.Version, Base64FormattingOptions.None);
                    if (!encodedNewTagValue.Equals(encodeExistingTagValue, StringComparison.Ordinal))
                    {
                        // concurrency conflict if the resource was modified
                        throw new HttpResponseException(HttpStatusCode.PreconditionFailed);
                    }
                }
            }
        }

        // POST /api/products
        public HttpResponseMessage Post(Product product)
        {
            _products.Add(product);
            var response = new HttpResponseMessage(product, HttpStatusCode.Created);
            response.Headers.Location = new Uri(Request.RequestUri,
                "/api/products/" + product.Id.ToString(CultureInfo.InvariantCulture));
            return response;
        }

    }

Liniile evidențiate arată părțile relevante:
– la GET, setăm ETag-ul resursei returnate folosind HttpRespnseMessage.Headers.ETag (valoarea trebuie să fie cuprinsă în ghilimele (“)
GET with ETag
– la PUT (modify), dacă this.Request.Headers.IfMatch are o valoare (și nu e “*”), o comparăm cu ETag-ul resursei persistate (în DB sau cache)
– dacă nu concid, returnăm un response cu HttpStatusCode.PreconditionFailed
PUT with ETag

Și câteva teste care ilustreză ceea ce ne așteptăm să se întâmple:

    [TestClass]
    public class ProductsControllerIntegrationTests
    {
        private HttpClient _client;
        private string _productUri = "http://ipv4.fiddler/TestETag/api/products/11";
        private readonly string jsonMediaType = "application/json";

        [TestInitialize]
        public void Init()
        {
            _client = new HttpClient();
        }

        [TestMethod]
        public async Task Get_ReturnsTheETagInTheHeaders()
        {
            HttpResponseMessage httpResp = await _client.GetAsync(_productUri);

            Assert.IsNotNull(httpResp.Headers.ETag, "No ETag was included in the response headers!");
            Assert.IsTrue(!string.IsNullOrEmpty(httpResp.Headers.ETag.Tag), "An empty ETag was received!");

        }

        [TestMethod]
        public async Task Put_IfTheProductWasModified_ReturnsPreconditionFailed()
        {
            // first user reads the product 11
            HttpResponseMessage getProdUser1 = await _client.GetAsync(_productUri);
            getProdUser1.EnsureSuccessStatusCode();
            Product prodUser1 = await getProdUser1.Content.ReadAsAsync();

            ///////////////
            // second user reads the product 11
            HttpResponseMessage getProdUser2 = await _client.GetAsync(_productUri);
            getProdUser2.EnsureSuccessStatusCode();
            Product prodUser2 = await getProdUser2.Content.ReadAsAsync();

            //second users modifies (PUT) and persist the product
            prodUser2.Description = prodUser2.Description + " modif by user 2";

            var putRequest2 = GetPutRequestMessage(prodUser2);
            // we have to use SendAsync in order to be able to set any header, including If-Match
            HttpResponseMessage putUser2Response = await _client.SendAsync(putRequest2);
            putUser2Response.EnsureSuccessStatusCode();

            ///////////////
            // first user modifies (PUT) and tries to persist the same product
            prodUser1.Description = prodUser2.Description + " modif by user 1";

            var putRequest1 = GetPutRequestMessage(prodUser1);
            HttpResponseMessage putUser1Response = await _client.SendAsync(putRequest1);

            /////////////////////
            Assert.AreEqual(HttpStatusCode.PreconditionFailed, putUser1Response.StatusCode,
                            "The response was not precondition failed!");

        }

        private HttpRequestMessage GetPutRequestMessage(Product product)
        {
            var requestMessage = new HttpRequestMessage(HttpMethod.Put, _productUri);
            // add an 'If-Match' header
            requestMessage.Headers.IfMatch.ParseAdd("\"" + Convert.ToBase64String(product.Version) + "\"");
            // add the modified object serialized as JSON
            ObjectContent prodObjContent =
                requestMessage.CreateContent(
                product,
                MediaTypeHeaderValue.Parse(jsonMediaType),
                new MediaTypeFormatter[] { new JsonMediaTypeFormatter() },
                new FormatterSelector());

            return requestMessage;
        }

        [TestCleanup]
        public void Cleanup()
        {
            if (_client != null)
            {
                _client.Dispose();
            }
        }

    }

Normal, nu contează limbajul sau frameworkul folosit pe client – am folosit HttpClient din ASP.NET Web API doar fiindcă era la îndemână, puteam folosi JavaScript, C++ sau altceva.

Pentru o aplicație reală, desigur ar mai fi câțiva pași: refactoring la greu pentru a obține pe cât posibil un cod reutilizabil, separarea codului ce face concurrency checks într-o clasă separată – separation of concerns (probabil într-un DelegatingChanell, precum e descris la http://javiercrespoalvez.com/2011/06/etags-and-optimistic-concurrency.html ), folosirea unei soluții de persitență reale (DB de ex.). De asemenea, evident testele de mai sus sunt doar niște integration tests, nu ar strica niște unit teste adevărate.

Ce nu e relevant in codul de mai sus:
– formatul folosit pentru ETag – am folosit byte[] doar fiindcă se mapează mai ușor la rowversion din SQL Server, când se folosește EF, dar la fel de bine putea fi un int sau Guid cât timp ne asigurăm ca e unic
– modul în care e encodat ETag-ul in header-ul HTTP: am folosit Convert.ToBase64String doar fiindcă e o modalitate comodă și safe de a encoda un array de bytes (în ambele direcții)
– formatul în care e serializată resursa (entitatea) – poate fi JSON ca mai sus, dar si XML sau altceva

Ce nu e abordat in acest post: rolul jucat de ETag in HTTP caching, care dacă e folosit devine relevant.

Posted in .NET, Web | Tagged , , , | 3 Comments

Berlin

Berlin_mar2012_010Berlin_mar2012_001Berlin_mar2012_002Berlin_mar2012_003Berlin_mar2012_005Berlin_mar2012_008
Berlin_mar2012_009Berlin_mar2012_011Berlin_mar2012_012Berlin_mar2012_013Berlin_mar2012_014Berlin_mar2012_015
Berlin_mar2012_016Berlin_mar2012_019Berlin_mar2012_020Berlin_mar2012_021Berlin_mar2012_022Berlin_mar2012_023
Berlin_mar2012_024Berlin_mar2012_025Berlin_mar2012_026Berlin_mar2012_027Berlin_mar2012_028Berlin_mar2012_029

Berlin, a set on Flickr.

Hoinărind prin Berlin..

Posted in Photos, Uncategorized | Tagged , | Leave a comment

CEBIT 2012

This gallery contains 23 photos.

Just some snapshots:

More Galleries | Tagged , | Leave a comment

Autocomplete “REST-style” în ASP.NET WebAPI

Să zicem că primim următorul task: să se creeze un HTTP service, care să poata fi apelat de un client oarecare pentru a implementa o funcționalitate gen auto-complete.
Poate că vrem ca acel service sa fie cât de cat “REST”-style: un request (GET) de genul
/…/resursa/name
ar trebui să returneze fie o singura resursă (in cazul in care e un singur match), fie lista resurselor al caror nume începe cu ‘name’ (și, normal 404 not found dacă nu se găsește nimic). Aparent, nimic complicat.
Totuși, dacă vrem ca al nostru serviciu sa fie “REST”-style, care ar trebui să fie status code-ul în cazul în care nu avem un match exact?
O variantă ar fi să returnăm un status code 300: ‘multiple choices’E potrivit acest status? S-ar putea – RFC-ul (http://tools.ietf.org/html/rfc2616#section-10) ne zice cam așa:

The requested resource corresponds to any one of a set of
representations, each with its own specific location, and agent-
driven negotiation information (section 12) is being provided so that
the user (or user agent) can select a preferred representation and
redirect its request to that location.
Unless it was a HEAD request, the response SHOULD include an entity
containing a list of resource characteristics and location(s) from
which the user or user agent can choose the one most appropriate. The
entity format is specified by the media type given in the Content-
Type header field. Depending upon the format and the capabilities of
the user agent, selection of the most appropriate choice MAY be
performed automatically. However, this specification does not define
any standard for such automatic selection.
If the server has a preferred choice of representation, it SHOULD
include the specific URI for that representation in the Location
field; user agents MAY use the Location field value for automatic
redirection. This response is cacheable unless indicated otherwise.
La prima vedere, nu e chiar cel mai potrivit răspuns – totuși – dacă citim un pic despre “agent-driven negotiation”, printre variante există și: “manually by the user selecting from a generated (possibly hypertext) menu” ceea ce ne apropie de un auto-complete menu.

Cum implementăm asta folosind ASP.NET WebAPI? Destul de simplu:

public class CitiesController : ApiController
 {
 private readonly string[] _cities = { "Cluj", "Bucuresti", "Iasi", "Constanta", "Călarasi", "Câmpeni", "Câmpia Turzii", "Câmpina", "Câmpulung", "Călan", "Călimanesti" };

// GET /api/cities/name
 public HttpResponseMessage<IEnumerable<string>> Get(string cityName)
 {
 var matches = (from c in _cities
                where c.StartsWith(cityName, StringComparison.CurrentCultureIgnoreCase)
                select c).ToArray();

HttpStatusCode status = HttpStatusCode.OK;

if (matches.Length == 0)
 {
   throw new HttpResponseException(HttpStatusCode.NotFound);
 }
   else if (matches.Length == 1)
 {
   status = HttpStatusCode.OK;
 }
 else if (matches.Length > 1)
 {
   status = HttpStatusCode.MultipleChoices;
 }

 return new HttpResponseMessage<IEnumerable<string>>(matches, status);
 }
 }

Clar, e doar un exemplu simplificat, într-o aplicație reala probabil ar mai fi și alți parametrii, validări, rezultatele ar fi extrase dintr-o baza de date etc..

Cum arată un astfel de response văzut cu Fiddler?

Fiddler capture - HTTP 300 status

Este aceasta cea mai bună abordare? Încă nu sunt sigur – e clar că depinde și de clientul ce va consuma acest serviciu (ex.: JQuery), dacă știe să interpreteze respectivul status code.

Privit din perspectiva “REST” justificarea ar fi: un GET pentru /api/cities/Că
dacă nu gaseste o singură resursă, ar trebui să returneze 404 not found.
300 ‘multiple choices’ ne permite să spunem clientului ceva de genul: “nu am găsit orașul respectiv, dar iată o listă de posibilități din care poti alege”.
E un “redirect” pentru că îl trimitem pe client să caute altundeva.
(totuși, nu e vorba de 301, 302 sau 307 care sunt folosite de obicei pentru redirectari permanente sau temporare)

Mai sus, am pus “REST” in ghilimele, pentru că se prea poate ca serviciul meu să nu fie cu adevărat REST, in sensul în care e definit în dizertatia lui Roy Fielding. De ex., lista cu opțiunile posibile, nu include și locația fiecăreia, deci clientul trebuie să știe cum să obțină un city ulterior.

Posted in .NET, Web | Tagged , , , | 3 Comments