New Posts New Posts RSS Feed: Ideablade is not unit test friendly
  FAQ FAQ  Forum Search   Calendar   Register Register  Login Login

Ideablade is not unit test friendly
 Post Reply Post Reply
Author
  Share Topic Share Topic  Topic Search Topic Search  Topic Options Topic Options
mgood View Drop Down
IdeaBlade
IdeaBlade
Avatar

Joined: 18-Nov-2010
Location: Emeryville, CA
Posts: 583 		Post Options Post Options   Quote mgood Quote  Post ReplyReply Direct Link To This Post Topic: Ideablade is not unit test friendly
    Posted: 10-Aug-2011 at 2:46pm

@brakowski,
Ward forwarded me this post. I'm the Senior Director of Professional Services and as Ward stated we do a lot of testing for our customers. I wanted to quickly follow up on the repository pattern and your concern around the BaseOperation and derived classes.

You are correct that BaseOperation is nearly impossible to fake, but BaseOperation implements an interface called INotifyCompleted through which you can interact with the operation and which in turn can easily be faked. We take advantage of this in the method signature of the repositories. I wanted to point you to a write-up on Repositories in the context of our DevForce-Caliburn framework. You might be interested in the whole framework as well.

http://devforcecaliburn.codeplex.com/wikipage?title=The%20Data%20Repository&referringTitle=Documentation

Our best practice approach for repositories hides the DevForce goop and abstracts the persistence layer not only for testing, but for the rest of the application as well.

Also, the HelloWorld sample shows a simple progression of tests. It goes to what Ward was saying. A lot of the testing can be done synchronously. The EntityManager is abstracted behind an EntityManagerProvider, which for testing returns an offline EM with sample data to do a lot of synchronous in memory testing and also shows how the tests can be taken to the next level and run against the fake backing store to have the almost end-to-end scenario using the same sample data. The fake store gets initialized with the sample data before the test, so you always have a known state of the data.
 
 
Testability goes beyond just DevForce and has more to do with how the entire application is structured and how the various layers are abstracted.


Edited by mgood - 11-Aug-2011 at 11:04am
Back to Top
WardBell View Drop Down
IdeaBlade
IdeaBlade
Avatar

Joined: 31-Mar-2009
Location: Emeryville, CA,
Posts: 338 		Post Options Post Options   Quote WardBell Quote  Post ReplyReply Direct Link To This Post Posted: 10-Aug-2011 at 12:57pm
Ahh ... Interaction testing! That's pretty easy too ... at least in my experience.
 
My approach is to attach test-oriented event handlers to the EntityManager which positively bristles with informative pre- and post- events.  You can often change inputs and outputs through these events to simulate test conditions. I concede that this is not as obvious as mocking with proxies and overrides to a virtual method but it has proved sufficient to my testing needs.
 
Aside: I've compared the eventing approach to mocking and eventing tends to yield shorter, clearer set-up code. It could be me but I don't find the various mocking syntaxes all that readable. Don't get me wrong ... you can't beat mocks for some test scenarios ... but state-based testing holds up well most of the time.
 
I also make extensive use of the repository (as explained above) which gives you complete freedom to instrument the interaction between application components and the supporting persistence components.
 
That reduces the problem to testing the repository ... which is where the aforementioned EntityManager eventing comes in.
 
Now you also want to know when things are happening to entities in cache or the cache itself quite apart from the effects of query and save. Events to the rescue again. You'll find plenty on the entity cache, the validation engine, and the EntityGroups. Each type in cache is goverened by an EntityGroup; you can listen for every individual import, add, query, remove, delete, clear, accept, reject, etc.
 
If you come across some activity of the EntityManager ... or of any DevForce component ... that you'd like to detect/verify and you can't figure out how ... please follow up with us.
 
Email is great when it comes to application specifics. I'm pleased to carry on in this space when we can be concrete without disclosing proprietary features of your application. 
 
You are joined by many others (I hope!) in desiring good test strategies for your application. This forum is the best place to help everyone until we can cover the subject properly in the DevForce Resource Center.
 
So I thank you again for your willingness to ask these great questions here.


Edited by WardBell - 10-Aug-2011 at 1:02pm
Back to Top
brakowski View Drop Down
Newbie
Newbie


Joined: 08-Aug-2011
Location: Warsaw, Poland
Posts: 3 		Post Options Post Options   Quote brakowski Quote  Post ReplyReply Direct Link To This Post Posted: 10-Aug-2011 at 4:15am
Thank you for a quick response.
At the moment we understand there is the proposed manner of testing and we are facing a decision whether the proposed method is suitable for us. For example: we can verify that the overall state of the EntityManager is right, but we can not verify that appropriate methods have been invoked...
... but I agree that testability is a big topic so I accept the suggestion to continue the conversation on the basis of concrete examples. I am here not for argument-fencing, but for the sake of a outcome of increased testability. I expect further conversation is to be continued by email.

Personal note: I'm also unavailable for the next two weeks, so I will carry on then.

Edited by brakowski - 10-Aug-2011 at 1:44pm
Back to Top
WardBell View Drop Down
IdeaBlade
IdeaBlade
Avatar

Joined: 31-Mar-2009
Location: Emeryville, CA,
Posts: 338 		Post Options Post Options   Quote WardBell Quote  Post ReplyReply Direct Link To This Post Posted: 09-Aug-2011 at 4:05pm
Hi @brakowski -
The least test-friendly aspect of DevForce is our lack of documentation on the subject. We have yet to describe how you can effectively and efficiently test your DevForce application. Explanations, samples, videos ... they're on our roadmap.
 
But the facts are: we make it easy to automate testing of DevForce applications - integration as well as unit testing.
Obviously testability is a big topic and would require far more space than I have for this reply. I'll offer some suggestions and point you to some examples along the way. Perhaps they will begin to change your opinion.
 
You are correct that many DevForce components do not expose interfaces that cover their public APIs. Many components expose abstract classes instead of interfaces and many methods you wish were virtual are not. This can be off-putting but it's not arbitrary. They are consequences of decisions we made in favor of making a stable and supportable commercial product. Those benefits come with trade-offs ... but this is a longer discussion for another time.
 
The good news is that, as a practical matter you don't always need these architectural artifacts. They are a means to an end. "Testing" is the end. Thanks to specific test-oriented features of DevForce, you can automate testing pretty easily once you know how.
 
Here are some brief clues:
 
Test your entities independently of DevForce components. You can "new" them and exercise your custom business logic with free-standing entity objects. While your entities do derive from our Entity base class, we don't impose a constructor on you and they work fine on their own ... until you engage DevForce related logic such as navigations, validations and property interception ... at which point ...
 
Add test entities to an offline "Test" EntityManager.
 
1. Create a test EntityManager
 
var testEm = new MyEntityManager(shouldConnect: false);
 
2. "New" up some entities, including related entities, and add them to the testEm
 
var testCustomer = new Customer {Name = "Galactic Wonder"};
testEm.AddEntity(testCustomer);
 
testCustomer.Orders.Add(new Order {OrderDate = DateTime.Now}); // Add orders to customer
testCustomer.Orders.Add(new Order {OrderDate = new DateTime(2011, 1, 1)});
 
testEm.AcceptChanges(); // Added entities are now "Unchanged"; simulating result of a query.
 
Now you're ready to play with a cache of test entities, configured as you need them. Navigation, validation, property interception work.
 
You can invoke queries on the testEm as if you expected them to go to the database. They'll run cache-only (unless you explicitly set their QueryStrategies) because an offline EntityManager queries and navigates cache-only.
 
Typically, this behavior will be transparent to the application components you're trying to test ... the ones that might be manipulating a Customer or issuing a query.
 
Use the Repository pattern. I like my application components to delegate all persistence operations to a client-side "repository" or "data service" that encapsulates the dirty details. My repositories have an interface which makes them easy to fake. My fake repositories often leverage an offline EntityManager (like testEm above) so that I get the effect of an in-memory database for free.
 
I use the repository to fake "saves" too. When the repository receives a save request, it manipulates the state of the cache to reflect the state you want the cache and entities to have in your test. For example, you could tweak some entity values and call "AcceptChanges()". You could throw an exception that appeared to have come from the server.
 
Note that throughout this process we've been working offline. We're not contacting the server at all! We don't even need a database.
 
This being Silverlight, your repostitory APIs must be structured as async calls. That usually makes testing hard. Not when you're using an offline test EntityManager. An EntityManager will return async calls synchronously when (a) it's offline or (b) it can satisfy the request entirely from cache.
 
I really like this offline approach because it means I don't have to engage the async aspects of the Silverlight Unit Test framework. That framework and the async testing style it imposes is painful.
 
It's also unavoidable when you're doing integration tests that touch the server. I strive to keep most of my tests local, offline, and synchronous. They're faster, cleaner and more robust than tests which depend upon database access ... especially async database access.
 
Use the DevForce Silverlight Async Test harness for integration tests that touch the server. We test Silverlight applications with the Silverlight Unit Test Framework (http://silverlight.codeplex.com/) which has special logic and patterns for asynchronous testing. One place to learn about it is an old post by Jeremy Likness, http://csharperimage.jeremylikness.com/2010/01/silverlight-unit-testing-framework.html.
 
Maybe you know about it already? In which case, you know about all that "Enqueue ..." jazz.
 
We've wrapped that in our test framework which rides on top of the Silverlight Unit Test Framework . Our wrapper framework tries to simplify the process AND also enables async testing of both Silverlight and regular .NET code. Yup ... you may want your WPF and Windows Forms clients to be async too. Our test harness can run the same async tests in Silverlight AND full .NET WITHOUT changing you test code.
 
You can see an example of what I mean in the "Querying" code sample on the DevForce Resource Center: http://drc.ideablade.com/xwiki/bin/view/Documentation/code-sample-querying
 
That's also a good place to see a variety of query testing patterns. Here's one example:
    [TestMethod, Asynchronous, Timeout(20000)]
    [Tag("Integration"), Tag("EasyQuery")]
    public void then_get_customers_with_Model_EntityManager() {
 
      DoItAsync(() =>
 
        Manager
          .Customers // "Get all customers" query
          .ExecuteAsync()
          .TestTheResults(
            custs => custs.ShouldNotBeEmpty("Should have customers"),
            "Customers query")
 
       );
    }
 
Fake Backing Store for (almost) end-to-end testing. Sometimes you want tests that exercise the path from the client, over the network, to the server and back. The only thing you DONT want to do is touch a database. Maybe you don't want to avoid database changes after the test (what a pain to unwind ... especially if the test fails midway). Perhaps you want to pre-fill the "database" with test data that's just for this one test only. Perhaps you don't even have a database.
 
DevForce has a "Fake Backing Store" to simulate a database. Read about it here: http://drc.ideablade.com/xwiki/bin/view/Documentation/discovery-compositioncontext-fake.
 
That' s also the place to learn about faking the LoginManager and faking other MEF-injected components.
 
 
The Base Operation and derivatives are not mockable
 
You're the second person to complain about that; I am the first :D.
 
Seriously, I tried to get that changed ... and I think it would be nice ... but when asked to prove that I really needed it ... I haven't been able to do that. I've always found an easy way to accomplish my testing goal, even if it wasnt' the first way I thought of.
 
Maybe you'll be the one with the compelling use case.
 
I don't understand what you're asking about "StateManager, CompositionContext". We don't have a "StateManager". Will you please explain what it is you wanted to test?
 
Conclusion:
 
I hope this post gets you started. I trust it shows we've given testing serious thought and dedication.
 
Our professional services team helps many of our customers write their real-world business applications and we do a lot of automated testing (unit and integration testing) for them.
 
So ... yes ... we know from long experience how easily you can test a DevForce application.
 
I am sure there are some areas that are hard to test. Maybe you've found one. We'd love to know about it.
 
Concrete examples are always critical. We could sit here all day and dream up things that might happen ... ways we might want to test something. But when we don't have a real use case, we leave such speculation in dreamland. Harsh experience has taught us: you don't get real work done supporting speculative cases.
 
If you're stuck, please give us something concrete that you need to test ... a small, clear example ... and we'll be happy to help you through it. We might learn something too that will help us improve the product for you ... and everyone.
 
Cheers!
 
Personal note: I'm out of the country for the next three weeks. Other folks at IdeaBlade may be able to answer your follow up questions on this subject in the interim.


Edited by WardBell - 09-Aug-2011 at 4:10pm
Back to Top
brakowski View Drop Down
Newbie
Newbie


Joined: 08-Aug-2011
Location: Warsaw, Poland
Posts: 3 		Post Options Post Options   Quote brakowski Quote  Post ReplyReply Direct Link To This Post Posted: 08-Aug-2011 at 11:58pm
We started testing the IdeaBlade libraries for Silverlight and .NET 4 in a small project in order to verify it's usability in the much larger projects. After a (longer) while we came to impression that this framework is not unit-test-friendly coded. There are many examples:
* EntityManager: it exposes an interface, but it is not usable one; it does not have virtual methods that could be mocked or overrided; it does not expose the interfaces for common operations (as login) so proper separation must be done by us; etc.
* LoginOperation, BaseOperation: classes which are handed over in events are not interface based, do not have virtual methods and as a result are hard to mock and even to create in the test context

* the same goes for StateManager, CompositionContext... etc.

Please do not get me wrong, I believe you did a great work building DevForce framework, but I want to ask one question: Are you planning to do something with the codebase to make it more suitable for isolation, mocking and unit testing?
Back to Top
 Post Reply Post Reply

Forum Jump Forum Permissions View Drop Down