Tag: Data Services
Url: http://wildermuth.com/downloads/wp7odatamovies.zip
While I was giving my OData talk, someone asked about consuming OData on the WP7 phone. I had done this on the CTP earlier, but hadn't tried it during the beta. So I figured I'd look into it today. While this is still pretty easy to do, the tooling still isn't in place. This means that you can't simply do an "Add Service Reference" to a Windows Phone 7 project. Instead you have to follow these steps:
- Download the Windows Phone 7 OData Library here.
- Unzip the Windows Phone 7 OData Library to get access to the reference.
- In your project, add a reference to the System.Data.Services.Client.dll assembly from the .zip file.
- Create the proxy classes by using the .NET 4.0's DataSvcUtil.exe (located in the %windir%\Microsoft.NET\Framework\v4.0.30319 directory). See the example below:
%windir%\Microsoft.NET\Framework\v4.0.30319\DataSvcUtil.exe
/uri:http://odata.netflix.com/Catalog/
/out:NetflixModel.cs
/Version:2.0
/DataServiceCollection
- Include this new proxy class (containing the context object and the data classes) in your WP7 application.
- Use the class to call out to the service as necessary:
NetflixCatalog ctx =
new NetflixCatalog(new Uri("http://odata.netflix.com/Catalog/"));
var qry = from p in ctx.People
.Expand("TitlesDirected")
where p.Name == "Stanley Kubrick"
select p;
// Create DataService query since we're not
// using a DataServiceCollection
var dsQry = (DataServiceQuery<Person>)qry;
dsQry.BeginExecute(r =>
{
try
{
var result = dsQry.EndExecute(r).FirstOrDefault();
if (result != null)
{
Deployment.Current.Dispatcher.BeginInvoke(() =>
{
Titles = result.TitlesDirected;
});
}
}
catch (Exception ex)
{
MessageBox.Show(ex.Message);
}
}, null);
Note that you need to use a Dispatcher as this call is *not* guaranteed to be called on the UI thread. This was the one piece that was unexpected. Once I marshaled the update to the UI thread, all worked perfectly.
Here's the example: download
Url: http://msdn.microsoft.com/en-us/library/ff84745...
As many of you may have heard, I recently launched http://GiveAQuiz.com as a new web site for creating and taking quizzes. The Data Team at Microsoft were great in helping me build this site. I've written a whitepaper detailing how we used the Microsoft data stack to accomplish it. This whitepaper shows how we used these data technologies to build the site:
- SQL Server 2008
- Entity Framework 4
- WCF Data Services 4
- Visual Studio 2010’s SQL Server Database Project
- SQL Server Reporting Services
If you are using Microsoft's Data stack for your own web-based and/or Silverlight site, give a look...hopefully some lessons learned you can use:
http://msdn.microsoft.com/en-us/library/ff847451.aspx
I've ported my XBoxGames Database (see this blog article for copies of the .mdf files) to SQL Azure and added OData support. You can find the feed here:
https://odata.sqlazurelabs.com/OData.svc/v0.1/w9hratewlg/XBoxGames
While this rocks. it did expose some of the lack of full OData support in SQL Azure but they're working on them immediately. Michael Pizzo is working on this support and send this list of what is and isn't working currently:
Supported:
Requests of the form “http://odata.sqlazurelabs.com/OData.svc/v0.1/[SQLAzureServerName]/[SQLAzureDatabaseName]” are recognized and correctly routed
- Supports PowerPivot, Astoria Client/ASR, LinqPad, etc.
Integration with Portal, ACS
- Impersonation based on security token for authenticated access
- Anonymous access mapped to a particular user (read-only reference data, browser access,…)
- Configuration information is read from a configuration table, hosted in sql azure
Metadata is dynamically loaded (changes to SqlAzure schema are immediately available through OData endpoint)
- Pluralization support for entityset naming
- IDs based on PKs
- Relationships identified based on FKs
- Goal is to generate an equivalent model to what EntityDesigner generates by default when doing modelgen from a database
The following requests are supported:
- ServiceDocument (/)
- metadata (/$metadata)
- entityset (/entitysetname)
- identity operator (/entitysetname(id))
- property navigation (/entityset(id)/propertyname)
- count (/$count)
- single segment relationship navigation (/entityset(id)/navigationpropertyname)
The following query operators are supported:
- $top
- $skip
- $orderby
- $filter with basic expression support
- $inlinecount
- $select
- $skiptoken
ServerDriven paging is supported
- first 50 records plus NextLink
The following are supported in expressions:
- math operators (add, subtract, multiply, divide, mod)
- comparison expressions (equal, notequal, greaterthan, greaterthanorequal, lessthan, lessthanorequal)
- logic expressions (AND, OR, NOT)
- A host of string functions (length, substring, left, right, trim, replace, remove, upper, lower, startswith, endswith, concat, indexof)
Basic updating support
- user-defined key values
- autoincrement key values
Not supported (yet):
Stored procs
Tables/views w/no primary key
Date and Math scalar functions
EntityClient support
$expand
Many:Many relationships
Multi-level relationship navigation
Media Link Entries
Other feature requests?
Other Bugs:
Not listed in Michael's notes are a couple of other issues i've run into (but i've reported them to him so I expect they're going to be addressed soon):
Filtering along relationships:
- https://odata.sqlazurelabs.com/OData.svc/v0.1/w9hratewlg/XBoxGames/Games?$filter=Genre_1/Name eq 'Shooter'
Date Filtering:
- https://odata.sqlazurelabs.com/OData.svc/v0.1/w9hratewlg/XBoxGames/Games?$filter=ReleaseDate gt datetime'2009-12-12T12:00'
Ordering by Dates:
- https://odata.sqlazurelabs.com/OData.svc/v0.1/w9hratewlg/XBoxGames/Games?$orderby=ReleaseDate
Url: http://wildermuth.com/downloads/dataservicejque...
I'd recently been asked by Chris Sells to help him with a simple WCF Data Services/jQuery example so I thought I'd share it via my blog as well. The basic idea is to use jQuery's AJAX functionality to retrieve JSON instead of the usual OData and consume it on a web page.
The example I decided on using is to expose the XBox database with paging. I am not doing any of the niceties like getting result counts to show a real navigation bar. Instead this is quick and dirty to simply do "next" and "prev" and use WCF Data Service's URI API to retrieve data.
Like most of my XBox examples, I am using the Entity Framework to simply expose three entities types (Games, Genres and Ratings). Then I utilize a WCF Data Service to expose these types for consumption via REST. My HTML is simple:
<h2>
XBox Games</h2>
<div>
<a href="#" id="PrevPage">Prev</a>
<a href="#" id="NextPage">Next</a>
</div>
<div id="gameTable" />
The gameTable will simply be filled with a simple TABLE element for our data. So to the jQuery we go. I am using plain jQuery 1.3.2 (though plugins could be used to simplify some of this code, I choose to just do it raw for simplicity).
First I set up a couple of variables to hold our paging information:
// globals for paging
var page = 0;
var pagesize = 25;
Next I handle the document's ready function to do some work when the initial page load is complete:
// Loads once the document has completed loading
$(document).ready(function () {
// Set up paging buttons
$('#PrevPage').click(function (evt) {
// Stop the navigation from happening
evt.preventDefault();
if (page > 0) {
page--;
loadData();
}
});
$('#NextPage').click(function (evt) {
evt.preventDefault();
page++;
loadData();
});
// Load the initial data
loadData();
});
When setting up the paging buttins, I use the '#PrevPage' and '#NextPage' CSS selectors to find items named PrevPage and NextPage and wire up the event to cause the page changes to happen. The 'evt.preventDefault()' call stops the buttons from trying to navigate since we're using them as buttons. FInally once we change the current page value, we call loadData (or in the initial case, call loadData with the default values).
function loadData() {
var url = "/GameService.svc/Games?$orderby=Name" +
"&$skip=" + (page * pagesize) +
"&$top=" + pagesize;
$.ajax({
type: "GET",
url: url,
contentType: "application/json; charset=utf-8",
dataType: "json",
success: function (msg) {
loadTable(msg.d);
}
});
}
In the loadData function, I first create the URL by using the Data Service/OData URL API to retrieve games by Name while using the global values to retrieve a page of data at a time (see the $skip and $top parameters). Once the URL is defined, jQuery's ajax function executes the URL and returns in the success function in which we call loadTable to create the table for us. Since the returned JSON is contained in a 'd' element for security, we de-reference the 'd' element before we sent it into the loadTable function.
function loadTable(results) {
var table = '<table><thead><tr><th>Name</th>' +
'<th>Publisher</th><th>Box</th></thead><tbody>';
for (var post in results) {
var row = '<tr>';
row += '<td>' + results[post].Name + '</td>';
row += '<td>' + results[post].Publisher + '</a></td>';
row += '<td><img src="' + results[post].ImageUrl +
'" style="width: 100px; height=150px" /></td>';
row += '</tr>';
table += row;
}
table += '</tbody></table>';
$('#gameTable').html(table);
}
Finally in loadTable some simple HTML construction goes on to create a table with our results. The for loop returns a iterator for each row in the collection which we can use to retrieve the individual item (see results[post].Name as an example). Once the table is built, it replaces the contents of the element named gameTable with the new table.
Because WCF Data Services will return JSON as well, you can use it to do AJAX work just as effectively as Silverlight can consume it for rich client work. You can get the code here:
http://wildermuth.com/downloads/DataServiceJQuery.zip
I've said much about my opinion of Silverlight data access. Currently this is Web Services, WCF Data Services and WCF RIA Services. Let's talk about Data Services and RIA Services and how they are related:
WCF Data Services
Data Services are a good story in Silverlight, but only as of version 2.0 of the protocol. For those users of version 1.0, using Data Services was painful as automatic tracking did not exist yet. Data Services can create not only the data contract classes, but a context object that will do tracking and batching for you. The basic story of how Data Services works is that it takes a context object from a LINQ provider and exposes all the IQueryable endpoints as REST resources that can be queried. This works well in creating a place to execute queries and post/put/delete changes. While Data Services' URI API is another skill that could be learned, the Silverlight (and .NET) client library allows you to just issue LINQ queries to the data service. In addition, the ability to create client-designed graphs (e.g. embed relationships for eager loading) and navigate relationships to the server presents a good API for a data-centric application today.
The method for connecting to a data service is via a "Add Service Reference..." which means that it is easy to have data services projects that are outside a particular solution file. This is critical for when you are building very large projects.
One of my favorite things about Data Services today is the ability to create tracked projections from the Silverlight client:
var qry = from g in ctx.Games
where g.ReleaseDate <= cutoffDate
orderby g.ReleaseDate
select new
{
Name = g.Name,
Id = g.Id,
ReleaseDate = g.ReleaseDate
};
This projection (which minimizes the data being sent over the wire from a Games entity), can be tracked like any other entity. This is a big boon for Silverlight clients where we want to reduce the surface area of the data being sent over the wire.
WCF Data Services' standard wire protocol is OData (which is a variant of ATOM). This means that tools like PivotTable, code-name Dallas and others can co-mingle data with other systems that are supporting OData like SharePoint, WebSphere, OpenGovernment project and SQL Azure table storage.
Some of the problems with Data Services include the fact that Service Operations are not part of the Silverlight proxy (and are not completely supported by the client library); any validation attributes of model information (like string length) are left on the client; and having to wrap your collections as DataServiceCollections to support tracking.
While I think Data Services still represents a major case for Silverlight usage, it still has a way to go before I am completely satisfied.
WCF RIA Services
WCF RIA Services takes a different approach to data access the resembles a mix of Data Services and Web Services. In RIA Services a server project is set up to expose data across the wire, but instead of Data Services' approach of exposing queryable end-points, it creates methods on the server which can be executed to retrieve, update, insert and delete data. These methods can be crafted to take specific parameters which takes some of the responsibility for what to query out of the hands of the developer (which may be fine in some cases). RIA Services does take model constraints and validation attributes and generate them in the client, which is a big win. Also, RIA Services is like Data Services in that you can still do basic sorting, filtering and paging via the service.
The way that RIA Services is wired up to a project (via a Shared Code folder that is generated as the server-side project changes) is sexy, but for large projects the need to keep the RIA Services' project in the same solution (or use a project file hack to get it working) becomes an issue.
RIA Services is really built for the RIA Services' DomainContext to be the center of the application (which you can see when you look at demos utilizing their Data Source object that automatically wires up data as XAML elements). While you can write a loosely coupled application, it takes planning and is not the natural design of the API.
Other issue I have is with the design of the Load API where the attempt is made to hide the asynchronicity of the call. The problem is that when errors happen on the load operation, it is hard to capture the error (other than letting it be cause as a top-level exception) so that most users will need to provide a callback which is just like having a full asynchronous operation. No better than before but the principle of least surprise isn't followed.
Finally, in comparison with Data Services, RIA Services has a lot of the same features but is missing the ability to do projections, client-defined eager loading and loading properties (e.g. relationships).
What does all this mean?
So here we sit as Silverlight developers and we are being asked by Microsoft to choose of the three major options for data access. The problem is that none of them are perfect by any stretch of the imagination. I would think that Data Services and RIA Services are both great solutions but they are both missing features the others have. In some ways the generation of these two projects has caused a lot of confusion as to which is the prescribed method for doing data access in Silverlight. It reminds me of the Entity Framework vs. LINQ to SQL debacle.
But where do we go from here? I'd like to implore Microsoft to fast-track the work needed to provide parity in these products. That means build the underpinnings of both projects from the same underlying platform. That way the features of each platform could be included in the other. The features do not become the discrimator but instead it is the style of data access. We want and need the missing features on both platforms:
- "Add Service Reference..." Linkage to support large development projects.
- Exposure and code-generation of any EDMX constraints and Validation attributes.
- Projection and tracked minimizing projection support.
- Full eager loading and navigation support.
- Operations and Endpoints have the same level of support in the generated code.
- Obvious and simple communication APIs to prevent the library from surprising us with error propagation.
...One More Thing
I also have the request that for the next development cycle that we provide secondary methods for communicating validation information than just validation attributes. Validation attributes cause our server-side POCO classes to be not really POCO (since they have to have a reference to the validation assembly). As well as most cases require us to use the painful and hacky MetadataClass method of creating a class to *only* hold these attributes. You don't have to get rid of the attributes, just let us feed them to the model in other ways (open the hooks, we'll decide how to communicate it; whether it be a DSL, XML file, database, etc.).
Your Job
As a reader of this blog, I'd like to ask you a favor. Respond to this blog entry. I want to make sure my frustration with the two data access stacks is not just my annoyance but is causing confusion and pain to you my readers. If you agree, just comment (it can be just saying "I agree") so I can make sure the Data team at Microsoft understands how it impacts Silverlight (and others) development efforts. Thanks!
If you've been following my blog, you should know that I am keeping a pretty close watch on ADO.NET Data Services. The team recently released a second CTP of the new version with some interesting features. This CTP has some pretty compelling additions, but I am going ot focus on one in particular.
I've been teaching and using ADO.NET Data Services for a long time and I like showing off exposing a LINQ-based provider (Entity Framework, NHibernate or others) to a Silverlight application. While ADO.NET Data Services does expose its API through a REST API, the magic for me is in its use in Silverlight. In case you haven't been following along, using the Silverlight client you can issue a LINQ query through the Silverlight client (though in fairness, the full power of LINQ is not supported in the client):
var qry = (from g in ctx.Games
where g.Genre.Name == "Shooter"
orderby g.ReleaseDate
select g) as DataServiceQuery<Game>;
qry.BeginExecute(new AsyncCallback(r =>
{
try
{
theList.ItemsSource = qry.EndExecute(r).ToList();
}
catch (Exception ex)
{
// NOOP
}
}), null);
This is a powerful feature so that (unlike web services) developers can use a looser service definition to define their data stack. Let the developer who needs data be able to sort, filter and shape that data as necessary.
The last part of that phrase is "shape" on purpose. When I first teach this technology, I am often asked "what sort of shaping do you mean?" In ADO.NET Data Services 1.0, you could shape it by returning a hierarchy of data using the Expand method in the LINQ query. For example, we can return the game data with the Genre entity attached to every game like so:
var qry = (from g in ctx.Games
.Expand("Genre")
where g.Genre.Name == "Shooter"
orderby g.ReleaseDate
select g) as DataServiceQuery<Game>;
qry.BeginExecute(new AsyncCallback(r =>
{
try
{
theList.ItemsSource = qry.EndExecute(r).ToList();
}
catch (Exception ex)
{
// NOOP
}
}), null);
But this level of shaping is powerful but limited in scope. This is where ADO.NET Data Services 1.5 comes up trumps. In this new CTP we can now do projections to retrieve specific data. For example:
var qry = (from g in ctx.Games
where g.Genre.Name == "Shooter"
orderby g.ReleaseDate
select new
{
Name = g.Name,
ReleaseDate = g.ReleaseDate.GetValueOrDefault(),
}) as DataServiceQuery;
This query creates a new anonymous type that has the two values we want instead of returning the entire entity. In any other .NET project this would be fine, but for Silverlight we can't use the anonymous type for data binding (for security reasons). So like other anonymous type uses in Silverlight, you must project them into a known type. A type like that is pretty easy to cruft up if you need:
var qry = (from g in ctx.Games
where g.Genre.Name == "Shooter"
orderby g.ReleaseDate
select new GameInfo()
{
Name = g.Name,
ReleaseDate = g.ReleaseDate.GetValueOrDefault()
}) as DataServiceQuery<GameInfo>;
This is useful to no only limit the fields you return but also to shape the result. For example we can use the projection to retrieve the name, release date and the genre's name (normally a related object) like so:
var qry = (from g in ctx.Games
where g.Genre.Name == "Shooter"
orderby g.ReleaseDate
select new GameInfo()
{
Name = g.Name,
ReleaseDate = g.ReleaseDate.GetValueOrDefault(),
GenreName = g.Genre.Name
}) as DataServiceQuery<GameInfo>;
This is all well and good, but what about saving these projections? For the most part you should treat complex projections (like the one above) as read-only. But if you retrieve entities that are purely subsets of existing entities, then ADO.NET Data Services will handle it. Though currently this is crashing in Silverlight (it is a CTP after all). I will update this if I find a way around the saving bug in the near future and post the demo at that time.
But in general this is an awesome new addition to the library that I hope will close some of the use-cases that forced me to write Service Operations in the past.
What do you think?
It all depends...
Url: http://wildermuth.com/downloads/DevTeachGameSto...
Fun first day here at DevTeach. I a talk showing creating a MVVM application from scratch (using ADO.NET Data Services 1.5 CTP). Kathleen Dollard had introduced the concept earlier that day, but we teach it a lot differently so it was fun to show off building a full MVVM application in the span of 75 minutes.
The project we built in the talk is now available here at:
I the demo is a little big because I included the database (that includes some additional information). You will need Silverlight 3, ADO.NET Data Services 1.5 CTP and to attach the database (as XBoxStore) to SQL Server Express to get the demo to work. I introduced the topic of separation and isolation of contract with interfaces. Hope it helps!
Url: http://notatpdc.com/presentations/
Today at 3pm CST (4pm EST, 1pm PST) I will be doing a LiveMeeting talk on ADO.NET Data Services. If you are not at the PDC this week, drop by NotAtPDC.com and check out my session!
There is a known problem with ADO.NET Data Services today that is important if you (or your server) lives in specific timezones. The problem is associated with the way that the Silverlight Data Services Library constructs their URI for searches.
The problem surfaces if you do a query that has a DateTime comparison in it. For example:
var qry = from o in ctx.Orders
where o.OrderDate <= dt
select o;
This query generates the following URI in the EST timezone in the US:
http://.../ProductService.svc/Orders()?$filter=OrderDate le datetime'2008-10-13T00:00:00-04:00'
This works great. The problem is that in other timezones (e.g. Bulgaria) where its forward of Greenwich Mean Time, so the UTC date is +03:00 like so:
http://.../ProductService.svc/Orders()?$filter=OrderDate le datetime'2008-10-13T00:00:00+03:00'
Because the "+" isn't URL Encoded, it becomes a space which makes the date incorrect. For now you can convert the date to universal time but that's a hack at best:
var qry = from o in ctx.Orders
where o.OrderDate <= DateTime.Today.ToUniversalTime()
select o;
It works but its a hack.
