ยปCore Development>Code coverage>Lib/test/libregrtest/cmdline.py

Python code coverage for Lib/test/libregrtest/cmdline.py

#countcontent
1n/aimport argparse
2n/aimport os
3n/aimport sys
4n/afrom test import support
5n/a
6n/a
7n/aUSAGE = """\
8n/apython -m test [options] [test_name1 [test_name2 ...]]
9n/apython path/to/Lib/test/regrtest.py [options] [test_name1 [test_name2 ...]]
10n/a"""
11n/a
12n/aDESCRIPTION = """\
13n/aRun Python regression tests.
14n/a
15n/aIf no arguments or options are provided, finds all files matching
16n/athe pattern "test_*" in the Lib/test subdirectory and runs
17n/athem in alphabetical order (but see -M and -u, below, for exceptions).
18n/a
19n/aFor more rigorous testing, it is useful to use the following
20n/acommand line:
21n/a
22n/apython -E -Wd -m test [options] [test_name1 ...]
23n/a"""
24n/a
25n/aEPILOG = """\
26n/aAdditional option details:
27n/a
28n/a-r randomizes test execution order. You can use --randseed=int to provide an
29n/aint seed value for the randomizer; this is useful for reproducing troublesome
30n/atest orders.
31n/a
32n/a-s On the first invocation of regrtest using -s, the first test file found
33n/aor the first test file given on the command line is run, and the name of
34n/athe next test is recorded in a file named pynexttest. If run from the
35n/aPython build directory, pynexttest is located in the 'build' subdirectory,
36n/aotherwise it is located in tempfile.gettempdir(). On subsequent runs,
37n/athe test in pynexttest is run, and the next test is written to pynexttest.
38n/aWhen the last test has been run, pynexttest is deleted. In this way it
39n/ais possible to single step through the test files. This is useful when
40n/adoing memory analysis on the Python interpreter, which process tends to
41n/aconsume too many resources to run the full regression test non-stop.
42n/a
43n/a-S is used to continue running tests after an aborted run. It will
44n/amaintain the order a standard run (ie, this assumes -r is not used).
45n/aThis is useful after the tests have prematurely stopped for some external
46n/areason and you want to start running from where you left off rather
47n/athan starting from the beginning.
48n/a
49n/a-f reads the names of tests from the file given as f's argument, one
50n/aor more test names per line. Whitespace is ignored. Blank lines and
51n/alines beginning with '#' are ignored. This is especially useful for
52n/awhittling down failures involving interactions among tests.
53n/a
54n/a-L causes the leaks(1) command to be run just before exit if it exists.
55n/aleaks(1) is available on Mac OS X and presumably on some other
56n/aFreeBSD-derived systems.
57n/a
58n/a-R runs each test several times and examines sys.gettotalrefcount() to
59n/asee if the test appears to be leaking references. The argument should
60n/abe of the form stab:run:fname where 'stab' is the number of times the
61n/atest is run to let gettotalrefcount settle down, 'run' is the number
62n/aof times further it is run and 'fname' is the name of the file the
63n/areports are written to. These parameters all have defaults (5, 4 and
64n/a"reflog.txt" respectively), and the minimal invocation is '-R :'.
65n/a
66n/a-M runs tests that require an exorbitant amount of memory. These tests
67n/atypically try to ascertain containers keep working when containing more than
68n/a2 billion objects, which only works on 64-bit systems. There are also some
69n/atests that try to exhaust the address space of the process, which only makes
70n/asense on 32-bit systems with at least 2Gb of memory. The passed-in memlimit,
71n/awhich is a string in the form of '2.5Gb', determines howmuch memory the
72n/atests will limit themselves to (but they may go slightly over.) The number
73n/ashouldn't be more memory than the machine has (including swap memory). You
74n/ashould also keep in mind that swap memory is generally much, much slower
75n/athan RAM, and setting memlimit to all available RAM or higher will heavily
76n/atax the machine. On the other hand, it is no use running these tests with a
77n/alimit of less than 2.5Gb, and many require more than 20Gb. Tests that expect
78n/ato use more than memlimit memory will be skipped. The big-memory tests
79n/agenerally run very, very long.
80n/a
81n/a-u is used to specify which special resource intensive tests to run,
82n/asuch as those requiring large file support or network connectivity.
83n/aThe argument is a comma-separated list of words indicating the
84n/aresources to test. Currently only the following are defined:
85n/a
86n/a all - Enable all special resources.
87n/a
88n/a none - Disable all special resources (this is the default).
89n/a
90n/a audio - Tests that use the audio device. (There are known
91n/a cases of broken audio drivers that can crash Python or
92n/a even the Linux kernel.)
93n/a
94n/a curses - Tests that use curses and will modify the terminal's
95n/a state and output modes.
96n/a
97n/a largefile - It is okay to run some test that may create huge
98n/a files. These tests can take a long time and may
99n/a consume >2GB of disk space temporarily.
100n/a
101n/a network - It is okay to run tests that use external network
102n/a resource, e.g. testing SSL support for sockets.
103n/a
104n/a decimal - Test the decimal module against a large suite that
105n/a verifies compliance with standards.
106n/a
107n/a cpu - Used for certain CPU-heavy tests.
108n/a
109n/a subprocess Run all tests for the subprocess module.
110n/a
111n/a urlfetch - It is okay to download files required on testing.
112n/a
113n/a gui - Run tests that require a running GUI.
114n/a
115n/a tzdata - Run tests that require timezone data.
116n/a
117n/aTo enable all resources except one, use '-uall,-<resource>'. For
118n/aexample, to run all the tests except for the gui tests, give the
119n/aoption '-uall,-gui'.
120n/a"""
121n/a
122n/a
123n/aRESOURCE_NAMES = ('audio', 'curses', 'largefile', 'network',
124n/a 'decimal', 'cpu', 'subprocess', 'urlfetch', 'gui', 'tzdata')
125n/a
126n/aclass _ArgParser(argparse.ArgumentParser):
127n/a
128n/a def error(self, message):
129n/a super().error(message + "\nPass -h or --help for complete help.")
130n/a
131n/a
132n/adef _create_parser():
133n/a # Set prog to prevent the uninformative "__main__.py" from displaying in
134n/a # error messages when using "python -m test ...".
135n/a parser = _ArgParser(prog='regrtest.py',
136n/a usage=USAGE,
137n/a description=DESCRIPTION,
138n/a epilog=EPILOG,
139n/a add_help=False,
140n/a formatter_class=argparse.RawDescriptionHelpFormatter)
141n/a
142n/a # Arguments with this clause added to its help are described further in
143n/a # the epilog's "Additional option details" section.
144n/a more_details = ' See the section at bottom for more details.'
145n/a
146n/a group = parser.add_argument_group('General options')
147n/a # We add help explicitly to control what argument group it renders under.
148n/a group.add_argument('-h', '--help', action='help',
149n/a help='show this help message and exit')
150n/a group.add_argument('--timeout', metavar='TIMEOUT', type=float,
151n/a help='dump the traceback and exit if a test takes '
152n/a 'more than TIMEOUT seconds; disabled if TIMEOUT '
153n/a 'is negative or equals to zero')
154n/a group.add_argument('--wait', action='store_true',
155n/a help='wait for user input, e.g., allow a debugger '
156n/a 'to be attached')
157n/a group.add_argument('--slaveargs', metavar='ARGS')
158n/a group.add_argument('-S', '--start', metavar='START',
159n/a help='the name of the test at which to start.' +
160n/a more_details)
161n/a
162n/a group = parser.add_argument_group('Verbosity')
163n/a group.add_argument('-v', '--verbose', action='count',
164n/a help='run tests in verbose mode with output to stdout')
165n/a group.add_argument('-w', '--verbose2', action='store_true',
166n/a help='re-run failed tests in verbose mode')
167n/a group.add_argument('-W', '--verbose3', action='store_true',
168n/a help='display test output on failure')
169n/a group.add_argument('-q', '--quiet', action='store_true',
170n/a help='no output unless one or more tests fail')
171n/a group.add_argument('-o', '--slowest', action='store_true', dest='print_slow',
172n/a help='print the slowest 10 tests')
173n/a group.add_argument('--header', action='store_true',
174n/a help='print header with interpreter info')
175n/a
176n/a group = parser.add_argument_group('Selecting tests')
177n/a group.add_argument('-r', '--randomize', action='store_true',
178n/a help='randomize test execution order.' + more_details)
179n/a group.add_argument('--randseed', metavar='SEED',
180n/a dest='random_seed', type=int,
181n/a help='pass a random seed to reproduce a previous '
182n/a 'random run')
183n/a group.add_argument('-f', '--fromfile', metavar='FILE',
184n/a help='read names of tests to run from a file.' +
185n/a more_details)
186n/a group.add_argument('-x', '--exclude', action='store_true',
187n/a help='arguments are tests to *exclude*')
188n/a group.add_argument('-s', '--single', action='store_true',
189n/a help='single step through a set of tests.' +
190n/a more_details)
191n/a group.add_argument('-m', '--match', metavar='PAT',
192n/a dest='match_tests',
193n/a help='match test cases and methods with glob pattern PAT')
194n/a group.add_argument('-G', '--failfast', action='store_true',
195n/a help='fail as soon as a test fails (only with -v or -W)')
196n/a group.add_argument('-u', '--use', metavar='RES1,RES2,...',
197n/a action='append', type=resources_list,
198n/a help='specify which special resource intensive tests '
199n/a 'to run.' + more_details)
200n/a group.add_argument('-M', '--memlimit', metavar='LIMIT',
201n/a help='run very large memory-consuming tests.' +
202n/a more_details)
203n/a group.add_argument('--testdir', metavar='DIR',
204n/a type=relative_filename,
205n/a help='execute test files in the specified directory '
206n/a '(instead of the Python stdlib test suite)')
207n/a
208n/a group = parser.add_argument_group('Special runs')
209n/a group.add_argument('-l', '--findleaks', action='store_true',
210n/a help='if GC is available detect tests that leak memory')
211n/a group.add_argument('-L', '--runleaks', action='store_true',
212n/a help='run the leaks(1) command just before exit.' +
213n/a more_details)
214n/a group.add_argument('-R', '--huntrleaks', metavar='RUNCOUNTS',
215n/a type=huntrleaks,
216n/a help='search for reference leaks (needs debug build, '
217n/a 'very slow).' + more_details)
218n/a group.add_argument('-j', '--multiprocess', metavar='PROCESSES',
219n/a dest='use_mp', type=int,
220n/a help='run PROCESSES processes at once')
221n/a group.add_argument('-T', '--coverage', action='store_true',
222n/a dest='trace',
223n/a help='turn on code coverage tracing using the trace '
224n/a 'module')
225n/a group.add_argument('-D', '--coverdir', metavar='DIR',
226n/a type=relative_filename,
227n/a help='directory where coverage files are put')
228n/a group.add_argument('-N', '--nocoverdir',
229n/a action='store_const', const=None, dest='coverdir',
230n/a help='put coverage files alongside modules')
231n/a group.add_argument('-t', '--threshold', metavar='THRESHOLD',
232n/a type=int,
233n/a help='call gc.set_threshold(THRESHOLD)')
234n/a group.add_argument('-n', '--nowindows', action='store_true',
235n/a help='suppress error message boxes on Windows')
236n/a group.add_argument('-F', '--forever', action='store_true',
237n/a help='run the specified tests in a loop, until an '
238n/a 'error happens')
239n/a group.add_argument('--list-tests', action='store_true',
240n/a help="only write the name of tests that will be run, "
241n/a "don't execute them")
242n/a group.add_argument('-P', '--pgo', dest='pgo', action='store_true',
243n/a help='enable Profile Guided Optimization training')
244n/a
245n/a return parser
246n/a
247n/a
248n/adef relative_filename(string):
249n/a # CWD is replaced with a temporary dir before calling main(), so we
250n/a # join it with the saved CWD so it ends up where the user expects.
251n/a return os.path.join(support.SAVEDCWD, string)
252n/a
253n/a
254n/adef huntrleaks(string):
255n/a args = string.split(':')
256n/a if len(args) not in (2, 3):
257n/a raise argparse.ArgumentTypeError(
258n/a 'needs 2 or 3 colon-separated arguments')
259n/a nwarmup = int(args[0]) if args[0] else 5
260n/a ntracked = int(args[1]) if args[1] else 4
261n/a fname = args[2] if len(args) > 2 and args[2] else 'reflog.txt'
262n/a return nwarmup, ntracked, fname
263n/a
264n/a
265n/adef resources_list(string):
266n/a u = [x.lower() for x in string.split(',')]
267n/a for r in u:
268n/a if r == 'all' or r == 'none':
269n/a continue
270n/a if r[0] == '-':
271n/a r = r[1:]
272n/a if r not in RESOURCE_NAMES:
273n/a raise argparse.ArgumentTypeError('invalid resource: ' + r)
274n/a return u
275n/a
276n/a
277n/adef _parse_args(args, **kwargs):
278n/a # Defaults
279n/a ns = argparse.Namespace(testdir=None, verbose=0, quiet=False,
280n/a exclude=False, single=False, randomize=False, fromfile=None,
281n/a findleaks=False, use_resources=None, trace=False, coverdir='coverage',
282n/a runleaks=False, huntrleaks=False, verbose2=False, print_slow=False,
283n/a random_seed=None, use_mp=None, verbose3=False, forever=False,
284n/a header=False, failfast=False, match_tests=None, pgo=False)
285n/a for k, v in kwargs.items():
286n/a if not hasattr(ns, k):
287n/a raise TypeError('%r is an invalid keyword argument '
288n/a 'for this function' % k)
289n/a setattr(ns, k, v)
290n/a if ns.use_resources is None:
291n/a ns.use_resources = []
292n/a
293n/a parser = _create_parser()
294n/a # Issue #14191: argparse doesn't support "intermixed" positional and
295n/a # optional arguments. Use parse_known_args() as workaround.
296n/a ns.args = parser.parse_known_args(args=args, namespace=ns)[1]
297n/a for arg in ns.args:
298n/a if arg.startswith('-'):
299n/a parser.error("unrecognized arguments: %s" % arg)
300n/a sys.exit(1)
301n/a
302n/a if ns.single and ns.fromfile:
303n/a parser.error("-s and -f don't go together!")
304n/a if ns.use_mp is not None and ns.trace:
305n/a parser.error("-T and -j don't go together!")
306n/a if ns.use_mp is not None and ns.findleaks:
307n/a parser.error("-l and -j don't go together!")
308n/a if ns.failfast and not (ns.verbose or ns.verbose3):
309n/a parser.error("-G/--failfast needs either -v or -W")
310n/a if ns.pgo and (ns.verbose or ns.verbose2 or ns.verbose3):
311n/a parser.error("--pgo/-v don't go together!")
312n/a
313n/a if ns.nowindows:
314n/a print("Warning: the --nowindows (-n) option is deprecated. "
315n/a "Use -vv to display assertions in stderr.", file=sys.stderr)
316n/a
317n/a if ns.quiet:
318n/a ns.verbose = 0
319n/a if ns.timeout is not None:
320n/a if ns.timeout <= 0:
321n/a ns.timeout = None
322n/a if ns.use_mp is not None:
323n/a if ns.use_mp <= 0:
324n/a # Use all cores + extras for tests that like to sleep
325n/a ns.use_mp = 2 + (os.cpu_count() or 1)
326n/a if ns.use:
327n/a for a in ns.use:
328n/a for r in a:
329n/a if r == 'all':
330n/a ns.use_resources[:] = RESOURCE_NAMES
331n/a continue
332n/a if r == 'none':
333n/a del ns.use_resources[:]
334n/a continue
335n/a remove = False
336n/a if r[0] == '-':
337n/a remove = True
338n/a r = r[1:]
339n/a if remove:
340n/a if r in ns.use_resources:
341n/a ns.use_resources.remove(r)
342n/a elif r not in ns.use_resources:
343n/a ns.use_resources.append(r)
344n/a if ns.random_seed is not None:
345n/a ns.randomize = True
346n/a
347n/a return ns