[Music] hello everyone welcome to this morning's session on test-driven development for the Android platform my name is Jonathan garish and I'm part of
the mobile ninjas we're a small team within Google who are passionate about software testing and can I get a quick show of hands in the audience how many of you are actually writing tests as part of your normal software development practice that's fantastic okay so if you've written tests for Android before you probably use some of our tools we developed the Android testing support library which includes the J unit for test run own rules the espresso UI testing framework and were also active contributors to robolectric the open-source Android unit testing framework so everyone's telling you to write tests but why should you really do it it's true that tests take time to write they're adding code to your codebase and perhaps you've been in a situation before where your manager or client has been telling you that they're slowing you down but there's so many compelling reasons to write tests tests give you rapid feedback on failures so failures that are spotted earlier on in the development cycle a far easier to fix than once they've gone live secondly tests give you a safety net you're free with a good suite of tests you're free to refactor clean up and optimize your code safe in the knowledge that you're not going to break existing behavior tests are really the backbone of sustainable software development you'll be able to maintain a stable velocity throughout the lifetime of your project and you're going to avoid the boom-bust cycles of crunch feature time and the aggregation of technical debt so in software testing there exists the concept of the pyramid the testing pyramid and this is made up of a number of layers and each layer brings with it its own trade-offs that you're going to have to weigh at the lowest layer is the small tests or the unit tests and these need to be very fast and highly focused that's why we recommended you these kind of tests is what is known as local unit tests and these are going to run on your local desktop machine and the trade-off you're making with these kind of tests is infidelity because you're not running on a realistic environment and you're probably substituting in a bunch of marks and fakes as we move up the pyramid we're now into the realms of integration testing and end-to-end testing and the key with these kind of tests is to bring in fidelity that's why we recommend that you run these kind of tests on a real device or an emulator these are the kind of tests that are going to tell you that your software actually works however they are less focused so a failure in one of these kind of tests might take a little longer to track down that it would in a unit test and one of the big trade-offs you're making is in test execution speed because you're assembling multiple components they all have to be built and then package shipped to a device where the tests are run and the results elected back that's going to take extra time so there's no single layer in this testing pyramid that can suffice so what you need to do is to blend in tests at each different tier leveraging the strengths of one category to way off the trade-offs in another there's no real hard and fast rule here but Google's own internal testing experts recommend the 70-20-10 rule of thumb as the ratio between small medium and large tests let's take a look at our workflow so and with test-driven development the idea is that you start by writing your tests then you implement a code to make those tests pass and then when your tests a green you can submit again a quick show of hands who out there has test-driven their code tried test-driven development in the past okay cool so we like test-driven development because it makes you to think about the design of your application up front it gives you consideration to api's and the structure of your code with test-driven development you're also going to be writing less code because you only write the code necessary to satisfy your tests this will enable you to release and often as you constantly green you'll be able to deploy a working application at a moment's notice if we're following the test pyramid the workflow is going to look something like this first of all we have a larger outer iteration that's concerned with feature development and here is started it's driven by a UI test and the mantra with test-driven development is red green refactor we start off with a failing test we implement the code to make that test pass and then we refactor inside the larger iteration are a series of smaller iterations and these are concerned with the unit tests here that you're assembling the building the units required to make the feature pass and again you use the same mantra here red green refactor red green refactor so let's take a look an example application so the feature going to implement today is the add notes flow to a sample note taking application if we take a look at our mock-ups we can see that we start on a notes list screen full of some existing notes there's a floating action button down at the bottom and the user will click this taking them on to the new add notes screen here they can enter a title and a description for their note before clicking save the node will be persisted and then never turned back to the notes list screen where they can see their newly added note along with any other notes that previously existed so coming back to our workflow for a moment remember that we start with a failing UI test so let's take a look at how this test would look like using espresso the UI testing framework the first step is to click on the add node button then we enter the title and description and click Save before returning to the notes list screen and here we're going to verify that the note that we just added actually shows up now remember with test-driven development we're not implement the code just yet and all we have to do is implement enough of the application to satisfy the specification of our tests so an empty activity and just the resources that we need will fyz once we have that we can run our test and we'll see it'll fail now we have to implement this feature so applications are built up of many small units these are small highly focused specialized components that do one thing and they do it well collections these small units are then assembled together so that they're collaborations will satisfy our feature let's take a moment to summarize the key characteristics of make up a good unit test as well as the normal conditions you wanting to start you wanting to test your failure conditions invalid inputs and boundary conditions you can end up writing a lot of unit tests unit tests must always give you the same result every time so avoid depending on things that might change for example an external server or the current time of date because this is going to bring flakiness into your unit tests unit tests should exercise one specific aspect of your code at a time you wanting to see that a failure in a unit test will lead you very quickly to an actual bug in your code and when you write unit tests avoid making too many assumptions on the actual implementation of your code you want your unit has to test behavior that way you avoid rewriting your tests when your implementations changing and one of the most important aspects of unit test is they've got to be fast especially because you're writing so many of them and you're in TDD workflow running them rapidly and it would it would be terrible if they were you were discouraged from writing tests or refactoring your code because of the pain in the execution time of those tests and finally unit tests are an excellent source of documentation and the way that's constantly evolving with your code as it changes unlike static documents that will stagnate over time let's write a unit test for our add notes activity this activity is going to take in user input and then we're going to persist it to local storage on the device okay so we're going to create the add note activity class and the select stand activity which is an android framework class it has a view which is going to be inflated with a layout and the user will enter their data here and then we're going to persist that note into Android shared preferences mechanism so it's conceivable that as our application evolves so does our requirement and perhaps our storage requirements evolve to persist the notes on to cloud storage and we have to build some kind of synchronization mechanism for local storage for the offline use case and in these cases we see opportunities for abstraction we might in this example see that we can extract a notes repository and however one of the key aspects of test-driven development is that we only start by writing the simplest case first and then we iterate so we're going to resist the temptation to do this early let's take a look at a sample of what an idealized unit test would look like they're generally built up into three stages so the first stage is you're setting the conditions for the test and this includes preparing the environment setting up your dependencies with a required state and preparing an input data next we'll exercise the code under test before finally making assertions on the results or the state so I like to clearly separate each of these three stages of the tests and bring the pertinent aspects of each test like front and center to make for really readable test so up until now with the Android platform you're writing your unit tests using the mockable jar in conjunction with a marking library such as mojito and let's take a look at an example of our test written with mojito okay wow that's a lot of code okay so because of we have so many interactions with the android framework and we're going to need to provide stubbing behavior for all of them in order just to make just to satisfy the execution paths of our tests and furthermore because Android uses a lot of static methods we're forced to introduce a second mocking library power mark that will handle this special case for us and there are also some pretty bad code smells here let's take a look you see we're forced to spy on the activity under test and we're needing to do this to modify its behavior and stubbing it out and providing some no ops so we're moving out of the realms of blackbox testing here and finally at the end we're making assertions about the implementation details and if these change our tests will need to change too so remembering the characteristics of a good unit test let's take a moment to score this particular test well it is very focused we're just testing the happy path of our add notes flow and it's certainly fast because it's running on the local JVM however we were making rather a lot of assumptions about the implementation in that test and and with this if any of our implementation changes it's likely we'll need to rewrite that test substantially and finally all that excess boilerplate stubbing is really distracting it's distracting away from the key aspects of the test the conditions the tests that you're trying to document well luckily there's a tool that helps address some of these issues so introducing Robo electric robolectric is an Android unit testing tool that's open source that we are actively contributing to and to tell you more about how you can write great tests with robolectric i'm going to hand you over to christian williams the original author of robolectric thanks Jonathan it's awesome to see so many people who are into Android testing and TDD so yeah rub electric is this a scrappy little open-source project that I started hacking on back in the early days of Android testing because I was just super annoyed at how long it took to to deploy and run tests on an emulator and it's kind of been a side project of a bunch of different people until last year when I had the privilege of joining my friend Jonathan at Google where he was already working on improving robolectric for Google's of internal test Suites and since then we've been we've been really beefing up for electric and contributing back to the open source project today robolectric isn't an officially supported part of the android plus testing platform but we found that when it's used correctly it can be a really useful part of your testing strategy and i'm going to show you a little bit about how you can how you can do that too so let's go back to our note student test and see how we might approach it with robolectric so since rubb electric runs as a local unit test it'll still be running in your workstation rather than an emulator but robolectric provides kind of a little Android sandbox next to your test where the actual SDK code is running you'll have access to your activities your layouts and views and resources and you can generally just call most Android methods and they'll kind of work like you'd expect there are parts of the android framework that rely on on native code or call it hardware or interact with external system services so for that robolectric provides a sort of test double that we call shadows and those provide alternative limitations of that code that's appropriate for unit testing so remember that that test that we just saw if it had like 20 lines of code of like mock set up code let's see how that looks in robolectric so that's a lot less we've gotten rid of all the boilerplate the test is about half the size and much more concise we're not forced to think about the implementation details as we're writing the test which is quite nice Reb logic is going to set up your application according to your manifest and here we're asking it to set up our activity it runs it through the appropriate life cycle to get it into the right state inflates views all that stuff it that we expect from on a device so we can just interact with it as if you're on the device so we add some texts and fields click on it and assert that it adds the note to the repository now notice that we're not actually going as far as the UI test that we saw that we wrote at the very beginning we're not asserting that the new note appears on a view screen that would be the job of another unit test now I mentioned robolectric shadows they actually give extended testing api's to some Android classes that let us let us query internal state and sometimes change your behavior in this example we're asking the application if any of our activities requested that an intent be launched during the test we could use that twister that after saving a note to the repository we're going to go to the to the view notes activity similar testing api's exists for simulating Hardware responses or external services things like that so at this point we have a failing unit test and now we get to we're ready for the easy part writing the production code in the spirit of TDD we're only going to write exactly as much as as is needed to make the test pass no more no speculative coding so we inflate a layout at acetic Handler and when the click happens we like for you to note an editor repository so now we can run the test see a pass if there's some improvement we can make to the code we'll go back and refactor and then we repeat this is where you get the thoroughness and Rev electric is super handy for this because it gives you like nice fast test runs you can get into a comfy cycle so we want to not discuss the happy path here we're going to test all the different cases we can there are codes likely to encounter so for example input validation and external conditions like the network being down stuff like that Rob let you can also help with simulating device conditions that you'll encounter for example you can specify qualifiers at the notes or that the test should run with here we're saying like a certain screen size and orientation which might change the layout a bit you can ask your lecturer to run your test under a specific SDK so we'll say jelly bean here and it actually uses the SDK code from that version and you can also tell Rob Electric I want to run this test under every SDK that you support or some some range of them that you're interested in and we support jelly bean through oh right now so at Google we rely really heavily on robolectric and we're investing making it better we've got dozens of apps including these that have hundreds of thousands of unit tests running internally so it's well battle tested and we've also recently started running the Android CTS which is a dandy official Android test suite again strobe electric and we're about 70 percent passing right now getting better with every release so if you've used our electric in the past and found that it's come up short or if you're stuck in an old version definitely recommend that you you get up to the latest because it's been a long way we've been working on reducing friction and then in integrating Rev electric with Android toolchain it works now very well with with Android studio with Gradle and we've got support for Basile Google's into Google's own open-source build system coming soon so robolectric isn't a one-size-fits-all testing tool it's fast but it's not a hundred percent identical to Android in every way so you want to use it judiciously as we said before avoid writing unit tests that link multiple activities together that's not so much a unit test that's much better for espresso if you find yourself dealing with multiple threads synchronization issues stuff like that you're also probably not writing a unit test so not good for electric and particularly avoid using robolectric to test reintegration with with android api s and things like Google Play services you really need to have higher level tests to give you confidence that that's working so now that we've got some passing unit tests I'm going to have you over to my colleague Stephan to talk about our level testing Thank You Christian so let's go back to our developer workflow diagram and so at this point we hopefully have a ton of unit tests and they thoroughly test all our business logic but let's switch gears and try to see how we can actually write some integration tests to see how these units integrate and how they actually integrate with Android and how they run in a real environment so on Android these tests are usually referred to as instrumentation tests and I'm pretty sure most of you have written an instrumentation test before and even though they look super simple on the surface there's actually a lot going on under the hood if you think about it you have to compile the code you have to process your resources you have to bring up a full system image and then run your tests and there's a lot of things that go on on various levels of the Android stack so these tests give you high fidelity but as John was mentioning they come at a cost which is they're slower and sometimes they're more flaky than unit tests so let's actually see how this works in your day-to-day development flow so let's say you're in Android studio you've just written your you know new espresso test and you hit the Run button to run the test so the first thing that Android studio is going to do is it's going to install two apks for you the test apk and the app under test so now the test APK contains Android JUnit runner it contains the test cases and your test manifest and then in order to run the test Android studio calls under the hood adb shell am instrument and then Android Jade runner will use instrumentation to control your app under test so what is instrumentation and I think you guys may have you guys may have noticed this it's a top-level tag in your manifest and why is that instrumentation is actually something that you know it's used deeply inside the android framework and it's used to control the life cycle of your activities for end so if you think about it it's a perfect interception point that we can use to inject the test runner and that's why Android drainage runner is not nothing more or less than an instrumentation so let's go a little bit deeper and see what happens when Android studio actually runs your tests so it runs adb shell am instrument which will end up calling out to activity manager activity manager will then call at one point on create on your instrumentation so now that we know that Android Jayne Runner is our instrumentation at this point it will call on create on the on the runner and then the runner is gonna do a few things for you it's going to collect all your tasks then it's going to run all these tests sequentially and then it's reporting back the results one thing to note here is that Android JUnit runner and you may have noticed this runs in the same process than your application and more importantly if you usually use Android join runner it runs all the tests in one single instrumentation invocation Android join each runner is heavily used inside Google we run billions of tests each month using Android JUnit runner and while doing so we saw some challenges that would that we face than if we had to solve one thing that we see a lot is shared state and I'm not talking about the kind of like shared state that you control and you that you code in your app I'm talking about the shared state that builds up on memory builds up on disk and makes you it has fail for you know no reason or you know unpredictable conditions and this among other things well at one point lead to crashes but in the previous model that I just showed you if if one of your tests crashes your instrumentation it will take the whole instrument the whole app process with it and all the subsequent tests will not run anymore and this is obviously a problem for large test suites similarly if you think about debugging if you run a couple of thousand tests in one invocation think about what your Lochhead will look like and when you have to go for it for debugging so that's why inside of Google we have taken a different approach so inside of Google every test runs in every test method runs in its own instrumentation invocation so now you can do this today right you can you know make multiple adb calls you can use a runner arc and maintain your custom script but the problem is it might not really integrate well with your development environment so that's why today I'm happy to announce the Android test Orchestrator and the Android test Orchestrator is a way that allows you to run tests like we do in Google it's a service ivk that runs in a background and runs each test in a single instrumentation invocation and this obviously has benefits right there's no shirt slate anymore and in fact the Android test Orchestrator runs p.
m.
clear before it runs its test more so crashes are now completely isolated because we have single instrumentation invocations if a crash happens all the subsequent tests will still run and similarly for debugging all the debugging information that you collect and pull off the device is now scoped to a particular test and this is great and this is great and we benefit a lot from it inside of Google so let's see how it actually works so on top of installing the test apk and upon our test what we do now is we install a third apk on the device and it's a service apk running in the background containing the orchestrator and then instead of running multiple adb commands we run a single ATB command but we don't instrument the app under test we instrument the orchestrator directly and then the orchestrator is going to do all its work on the device so it's going to use android change runner to collect your tests but then it's going to run each of those tests in its own invocation and it's amazing and I'm pretty sure you will you will like this a lot and it will be available in the next Android testing support library release and more importantly we will have integration with Android studio it will be available in Gradle and we will also have integration with firebase test lab coming later this year so now that we know how to run our tests let's actually look at how we can write these integration tests and usually if you write a UI test on Android you're using the espresso testing framework and as you can see espresso has this like nice and simple API and it actually works pretty simple what it does is you give us a view matcher and we find a view in the hierarchy that matches that matcher and then we either perform a view action or check a view assertion and because this API is so simple it's the perfect tool to for fast TDD prototyping of UI tests but in order to provide you such a simple API there's a lot of things that need to go on under the hood so let's actually look how espresso let's look at how spresso works so when you call on view and give us your matcher the first thing that we're going to do is we're going to create a view interaction for you and then the next thing is we make sure that your app is in an idle insane state before we are ready to interact with it and this is you can think of it this is at the core of espresso and espresso is well-known for it like synchronization guarantees and the way we do it is we loop the message queue until there are no messages for a reasonable amount of time we look at all your Eiling resources and make sure they're idle and we also look at async tasks to make sure there is no background work running and only if we know that your app is in a sane and stable State and we're ready to interact we're going to move on and then we're going to traverse the view hierarchy and find the view that matches your matcher and once we have the view we then going to perform a view action or a view certian and this is great so now let's circle back to the test that we showed you in the beginning and have a closer look now that we know how its pressure works so in the first line as you may remember we try to click on the add node button and here we just going to use a width ID matter which is a simple matter that is matching a view in the view hierarchy according to its ID and next thing we want to do is we want to click on the view and we use a click view action for this so now where it gets interesting is the next line because on this line we want to type the title and description and we want to use a type text action for that but here all the espresso synchronisation guarantees will kick in and only if we know that we're ready to interact with your implication we're going to invoke the type test action and this is great because it frees you from adding you know additional boilerplate code and additional slipping code to your test so similarly we're going to save the node and then we're going to verify that it's displayed on screen and this is great now we know how espresso works and we know how it's a great tool to do test-driven development and now I'm going to hand over to Nick to talk a little bit more on how you can improve your UI test and how to improve your large and medium testing strategy thank you so fun so one good attribute of a UI test is a test that never sleeps so let's go back to our example to illustrate this point a little bit further so in our examples you remember we have a note that we save into memory which is pretty fast and pretty reliable however reliably reality as your app grows you probably want to extend this functionality and save your note to the cloud or Google Drive for example so when running our large end-to-end test we want to use a real environment where we hit the real server and depending on your network connection this may take a long time so you probably want to do is in the background now the problem is now is that espresso synchronization is not aware of any of your long-running tasks so this is somewhere that some where developers will probably do something as ugly as putting a thread sleep in their code but what this presser is not actually required because you can write an idling resource where an idling resource is a simple interface for you as a developer to implement to teach espresso synchronization of any of your custom long running tasks of your app so with this ionic resource we made our large end-to-end test more reliable so let's see how we can add some more medium-sized stats to your test suite so for a medium-sized test we want to keep them small and focused on a single UI component whereas single UI component may be like a specific view fragment or an activity so let's go back to our example to see how we can isolate our large end-to-end test to more isolated components so here in this example again you may have noticed that there are two activities the list activity on the left and the add node activity on the right so until now we wrote a large end-to-end test that gives us a lot of confidence because it touches upon a lot of your code in your app which is great for large intent test but it's not so great for an iterative test-driven development cycle so so let's see how we can isolate these and have isolated tests for each activity in isolation to isolate the left-hand side the list activity we can use espresso intense we're espresso intense is a simple API that allows you to intercept any of you are going a tense verify their content and provide back a mock activity result great so let's see how that API actually looks like so as you can see it's very straightforward you have an intent matcher that will match your going intent and you can provide a version of your activity result back to the caller okay so let's use this API to write our first isolated test so in this test you can see on the first line we do exactly that we intercept our intent and we provide a stub version of our activity result now on the second line when we perform click instead of starting a new activity espresso will intercept this intent and provide a subjective result which we can then use on the last line to verify that our UI was updated accordingly now we have an isolated test okay so let's go back to our example and see how we can isolate the second part right so when you write when usually write tests you end up in a position where you may have some external dependencies in play that can check that are outside of your control so in our example right as I showed before we have I know that we save and it hits the real server now even though we have an auditing resource now that makes it more reliable your tests can still fail because your server may crash for some reason so your test will fail so wouldn't be better if we completely isolate ourselves from these conditions and run our tests in a hermetic environment this will not only make your test run much faster but it will also eliminate any flakiness and beyond this specific example you further want to isolate yourself from any external dependencies so for example you don't want to test any Android system UI or any other UI components that you don't own because they probably already tested and they can also change without your knowing so your tests will actually fail so let's see how our second isolated test will look like in code so here we see the main point here is that we no longer use the real server and instead you know we set up a hermetic repository now there's many different ways of you to do this and this is just one way so then you can use this harmonic repository in order to verify that your note is actually saved without ever leaving the context of your app or hitting the network so at this point if you think about it you have two smaller tests that run much more they are way more reliable and run much faster but at the same time you maintain the same amount of test coverage as your large end-to-end test and this is why we want to have more of these smaller isolated tests compared to the large end-to-end tests we showed before okay so at this point we iterated through our developer cycle a few times and we should see all of our tests start turning green and we should be confident to release our feature however before we conclude let's jump into the future for a second as your app grows and your team grows you continue adding more and more features to your app and you may find yourself in a position where you may have UI running in multiple processes which is exactly what happened at Google so if you go to our notes example this may look something like this you have a first activity that runs in your main process on the left-hand side and now the second activity will run in us in a private process and in this case we're going to call it add notes so how do we test that well before Android oh it was impossible to test but with Android oh there is a new instrumentation attribute that you can use in order to define which process you want to instrument while instrumenting and running tests against each process in isolation is a great idea and you should do it you may find yourself in a position where you want to cross process boundaries within one test so you would probably want to write an espresso test like looks like this while this was not only possible in a framework level before Android oh this was also impossible on this first level because in this specific example Express is not even aware of your secondary process nor can it maintain any of the synchronization guarantees we all know and love so today I'm happy to announce multi-process espresso support without changing any of your test code or your app code this will allow you to seamlessly interact with UI cross process while maintaining all of us presto synchronization guarantees and it will be able available in the next version of Android test support library release so let's have a quick overview of how it actually works so traditionally as you know in our example we start in one process where we have an instance of an original runner and espresso in this case now if you remember from our example when you click the add node button there will be a new activity and now we have a new process so the problem now is that we have two processes with two different instances of an original runner and espresso and they're not aware of each other so the first thing that we want to do is we want to establish communication between the two and regenerate runner and now that we have this communication we can use it to establish the communication video to espresso instances and the way we do that is by having an ability in hydrogen as runner to register any testing frameworks like espresso with an original runner and then the runner will then facilitate all the handshaking required in order to establish communication between the two espresso instances now that the two inspiration instances can talk to each other it can then use it in order to enable cross process testing and maintaining all the synchronization guarantees that we had before okay so with that we're reaching the end of our developer workflow and we showed you all the tools that you can use across each step of the way in order to make TDD happen and on Android and with that said even if you don't follow this flow exactly hopefully you know how to use every single tool and how to write good tests in order to bring your app quality to the next level so if you like to write tests and you want to write an on test like we do at Google here are some resources to get you started but I want to thank you and I think we have some time for questions and if not we we have office hours at 3:30 today so hopefully you see that thank you [Applause] [Music].
Không có nhận xét nào:
Đăng nhận xét