Device Farm is an app testing service that enables you to test and interact with your Android, iOS, and Web apps on real, physical phones and tablets that are hosted by Amazon Web Services (AWS). There are two main ways to use Device Farm:
- Automated testing of apps using a variety of available testing frameworks
- Remote access of devices onto which you can load, run, and interact with apps in real time
Device Farm allows you to upload your own tests or use built-in, script-free compatibility tests. Because testing is automatically performed in parallel, tests on multiple devices begin in minutes.
A test report containing high-level results, low-level logs, pixel-to-pixel screenshots, and performance data is updated as tests are completed.
Device Farm supports testing of native and hybrid Android, iOS, and Fire OS apps, including those created with PhoneGap, Titanium, Xamarin, Unity, and other frameworks. It also supports remote access of Android apps for interactive testing.
Device Farm currently provides support for the following test types:
Note: For the development of the automated tests, we used Appium JUnit framework for â€śCalculatorâ€ť application on android device.
Prepare Android Appium Java JUnit Tests for running on AWS Device Farm
Android Appium Java JUnit tests must be contained in a .zip file. POM file needs to be set up according to the official AWS Device Farm documentation forÂ JUnit.
Following dependencies should be added in maven POM file:
Go into your Appium Maven project directory from command prompt and run the following command to package the test content.
mvn clean package -DskipTests=true
After execution of this command, â€śzip-with-dependencies.zipâ€ť file will be created in the target folder, which will contain .jar files of the test classes and all dependency .jar files.
Note: â€śzip-with-dependencies.zipâ€ť file needs to be uploaded when creating a run on AWS Device Farm.
Create test run on AWS Device Farm
- Sign in to the Device Farm console atÂ https://console.aws.amazon.com/devicefarm
2. Click on â€śCreate a new projectâ€ť, enter project name and click on â€śCreate projectâ€ť button.
3. Click on â€śCreate a new runâ€ť button on project page.
4. On the â€śChoose your applicationâ€ť page, choose eitherÂ Native application (the Android and Apple button)Â or Web application (the HTML5 button). Upload your application file and click onÂ the â€śNext stepâ€ť button.
5. On Configure a test page, select test type, ex. â€śAppium Java JUnitâ€ť, upload â€śzip-with-dependencies.zipâ€ť file. It will take a few minutes for tests upload, after the test upload is finished select â€śAppium versionâ€ť and click on â€śNext stepâ€ť button.
6. Select the device (ex. Huawei P9) on â€śSelect devicesâ€ť and click on â€śNext stepâ€ť button.
7. Click on â€śNext stepâ€ť button on â€śSpecify device stateâ€ť page.
8. On â€śReview and start runâ€ť page set the execution timeout per device â€śex. 10 minutesâ€ť and click on â€śConfirm and start runâ€ť.
9. A report in Device Farm contains information about a run, which is a request for Device Farm to test a single app against one or more devices. Reports in Device Farm contain pass and fail information, crash reports, test and device logs, screenshots, and performance data. Reports include both detailed per-device data as well as high-level results, such as the number of occurrences of a given problem.
9.1 The â€śVideoâ€ť section displays a downloadable video recording of the test.
9.2 The â€śSuitesâ€ť section displays all test steps that contain pass and fail information and time of execution for each test step.
9.3 The â€śLogsâ€ť section displays any information Device Farm logged during the test.
9.4 TheÂ â€śPerformanceâ€ťÂ section displays information about any performance data Device Farm generated during the test.
AWS Device Farm is easy to use and offers a very detailed view of the results, providing all information needed to reproduce the bug. The main advantage is the possibility to execute tests on a wide range of devices, allowing us to discover problems, which canâ€™t be seen on the local test devices.