test: Add a README
Add a few notes about how testing works in U-Boot. Signed-off-by: Simon Glass <sjg@chromium.org> Reviewed-by: Teddy Reed <teddy.reed@gmail.com>
This commit is contained in:
		
							parent
							
								
									022885cb9c
								
							
						
					
					
						commit
						f6349c3c4c
					
				|  | @ -0,0 +1,82 @@ | ||||||
|  | Testing in U-Boot | ||||||
|  | ================= | ||||||
|  | 
 | ||||||
|  | U-Boot has a large amount of code. This file describes how this code is | ||||||
|  | tested and what tests you should write when adding a new feature. | ||||||
|  | 
 | ||||||
|  | 
 | ||||||
|  | Sandbox | ||||||
|  | ------- | ||||||
|  | U-Boot can be built as a user-space application (e.g. for Linux). This | ||||||
|  | allows test to be executed without needing target hardware. The 'sandbox' | ||||||
|  | target provides this feature and it is widely used in tests. | ||||||
|  | 
 | ||||||
|  | 
 | ||||||
|  | Pytest Suite | ||||||
|  | ------------ | ||||||
|  | 
 | ||||||
|  | Many tests are available using the pytest suite, in test/py. This can run | ||||||
|  | either on sandbox or on real hardware. It relies on the U-Boot console to | ||||||
|  | inject test commands and check the result. It is slower to run than C code, | ||||||
|  | but provides the ability to unify lots of test and summarise their results. | ||||||
|  | 
 | ||||||
|  | You can run the tests on sandbox with: | ||||||
|  | 
 | ||||||
|  | 	./test/py/test.py --bd sandbox --build | ||||||
|  | 
 | ||||||
|  | This will produce HTML output in build-sandbox/test-log.html | ||||||
|  | 
 | ||||||
|  | See test/py/README.md for more information about the pytest suite. | ||||||
|  | 
 | ||||||
|  | 
 | ||||||
|  | tbot | ||||||
|  | ---- | ||||||
|  | 
 | ||||||
|  | Tbot provides a way to execute tests on target hardware. It is intended for | ||||||
|  | trying out both U-Boot and Linux (and potentially other software) on a | ||||||
|  | number of boards automatically. It can be used to create a continuous test | ||||||
|  | environment. See tools/tbot/README for more information. | ||||||
|  | 
 | ||||||
|  | 
 | ||||||
|  | Ad-hoc tests | ||||||
|  | ------------ | ||||||
|  | 
 | ||||||
|  | There are several ad-hoc tests which run outside the pytest environment: | ||||||
|  | 
 | ||||||
|  |    test/fs	- File system test (shell script) | ||||||
|  |    test/image	- FIT and lagacy image tests (shell script and Python) | ||||||
|  |    test/stdint	- A test that stdint.h can be used in U-Boot (shell script) | ||||||
|  |    trace	- Test for the tracing feature (shell script) | ||||||
|  |    vboot	- Test for verified boot (shell script) | ||||||
|  | 
 | ||||||
|  | The above should be converted to run as part of the pytest suite. | ||||||
|  | 
 | ||||||
|  | 
 | ||||||
|  | When to write tests | ||||||
|  | ------------------- | ||||||
|  | 
 | ||||||
|  | If you add code to U-Boot without a test you are taking a risk. Even if you | ||||||
|  | perform thorough manual testing at the time of submission, it may break when | ||||||
|  | future changes are made to U-Boot. It may even break when applied to mainline, | ||||||
|  | if other changes interact with it. A good mindset is that untested code | ||||||
|  | probably doesn't work and should be deleted. | ||||||
|  | 
 | ||||||
|  | You can assume that the Pytest suite will be run before patches are accepted | ||||||
|  | to mainline, so this provides protection against future breakage. | ||||||
|  | 
 | ||||||
|  | On the other hand there is quite a bit of code that is not covered with tests, | ||||||
|  | or is covered sparingly. So here are some suggestions: | ||||||
|  | 
 | ||||||
|  | - If you are adding a new uclass, add a sandbox driver and a test that uses it | ||||||
|  | - If you are modifying code covered by an existing test, add a new test case | ||||||
|  |   to cover your changes | ||||||
|  | - If the code you are modifying has not tests, consider writing one. Even a | ||||||
|  |   very basic test is useful, and may be picked up and enhanced by others. It | ||||||
|  |   is much easier to add onto a test - writing a new large test can seem | ||||||
|  |   daunting to most contributors. | ||||||
|  | 
 | ||||||
|  | 
 | ||||||
|  | Future work | ||||||
|  | ----------- | ||||||
|  | 
 | ||||||
|  | Converting existing shell scripts into pytest tests. | ||||||
		Loading…
	
		Reference in New Issue