From c5db190f399bc8b4b600ca32c17adbd786ecd42b Mon Sep 17 00:00:00 2001 From: Zack Weinberg Date: Tue, 22 Dec 2015 12:30:58 -0500 Subject: [PATCH] Test directory reorganization. The `certs`, `fixtures`, `node_modules`, and `www` directories, and `testharness.js`, are moved into a new `lib` directory to reduce clutter; the long-term plan is that all other subdirectories will contain tests. (Right now, we still have `ghostdriver-tests` as an exception.) Adjust `writing-tests.md`, `run-tests.py`, `testharness.js`, and a couple of tests accordingly. Also fix a bug in `run-tests.py` where ERROR conditions that happened before the first test were not being reported accurately. --- test/{ => lib}/certs/https-snakeoil.crt | 0 test/{ => lib}/certs/https-snakeoil.key | 0 test/{ => lib}/fixtures/dummy.js | 0 test/{ => lib}/fixtures/error-helper.js | 0 test/{ => lib}/fixtures/parse-error-helper.js | 0 test/{ => lib}/node_modules/dummy_exposed.js | 0 test/{ => lib}/node_modules/dummy_file.js | 0 test/{ => lib}/node_modules/dummy_file2.js | 0 test/{ => lib}/testharness.js | 7 ++++ test/{ => lib}/www/__init__.py | 0 test/{ => lib}/www/delay.py | 0 test/{ => lib}/www/echo.py | 0 test/{ => lib}/www/frameset/frame1-1.html | 0 test/{ => lib}/www/frameset/frame1-2.html | 0 test/{ => lib}/www/frameset/frame1.html | 0 test/{ => lib}/www/frameset/frame2-1.html | 0 test/{ => lib}/www/frameset/frame2-2.html | 0 test/{ => lib}/www/frameset/frame2-3.html | 0 test/{ => lib}/www/frameset/frame2.html | 0 test/{ => lib}/www/frameset/index.html | 0 test/{ => lib}/www/hello.html | 0 test/{ => lib}/www/iframe.html | 0 test/{ => lib}/www/includejs.js | 0 test/{ => lib}/www/includejs1.html | 0 test/{ => lib}/www/includejs2.html | 0 test/{ => lib}/www/js-infinite-loop.html | 0 test/{ => lib}/www/logo.html | 0 test/{ => lib}/www/logo.png | Bin test/{ => lib}/www/missing-img.html | 0 test/{ => lib}/www/navigation/dest.html | 0 test/{ => lib}/www/navigation/index.html | 0 test/{ => lib}/www/phantomjs.png | Bin .../www/regression/pjs-10690/Windsong.ttf | Bin .../www/regression/pjs-10690/font.css | 0 .../www/regression/pjs-10690/index.html | 0 .../www/regression/pjs-10690/jquery.js | 0 .../www/regression/pjs-13551/child1.html | 0 .../www/regression/pjs-13551/child1a.html | 0 .../www/regression/pjs-13551/child2.html | 0 .../regression/pjs-13551/closing-parent.html | 0 .../pjs-13551/reloading-parent.html | 0 .../www/regression/webkit-60448.html | 0 test/{ => lib}/www/render/image.jpg | Bin test/{ => lib}/www/render/index.html | 0 test/{ => lib}/www/status.py | 0 test/{ => lib}/www/url-encoding.py | 0 test/{ => lib}/www/user-agent.html | 0 test/module/webpage/capture-content.js | 4 +-- test/module/webpage/on-error.js | 10 +++--- test/regression/pjs-13551.js | 2 +- test/run-tests.py | 28 ++++++++++++--- test/writing-tests.md | 34 +++++++++--------- 52 files changed, 54 insertions(+), 31 deletions(-) rename test/{ => lib}/certs/https-snakeoil.crt (100%) rename test/{ => lib}/certs/https-snakeoil.key (100%) rename test/{ => lib}/fixtures/dummy.js (100%) rename test/{ => lib}/fixtures/error-helper.js (100%) rename test/{ => lib}/fixtures/parse-error-helper.js (100%) rename test/{ => lib}/node_modules/dummy_exposed.js (100%) rename test/{ => lib}/node_modules/dummy_file.js (100%) rename test/{ => lib}/node_modules/dummy_file2.js (100%) rename test/{ => lib}/testharness.js (99%) rename test/{ => lib}/www/__init__.py (100%) rename test/{ => lib}/www/delay.py (100%) rename test/{ => lib}/www/echo.py (100%) rename test/{ => lib}/www/frameset/frame1-1.html (100%) rename test/{ => lib}/www/frameset/frame1-2.html (100%) rename test/{ => lib}/www/frameset/frame1.html (100%) rename test/{ => lib}/www/frameset/frame2-1.html (100%) rename test/{ => lib}/www/frameset/frame2-2.html (100%) rename test/{ => lib}/www/frameset/frame2-3.html (100%) rename test/{ => lib}/www/frameset/frame2.html (100%) rename test/{ => lib}/www/frameset/index.html (100%) rename test/{ => lib}/www/hello.html (100%) rename test/{ => lib}/www/iframe.html (100%) rename test/{ => lib}/www/includejs.js (100%) rename test/{ => lib}/www/includejs1.html (100%) rename test/{ => lib}/www/includejs2.html (100%) rename test/{ => lib}/www/js-infinite-loop.html (100%) rename test/{ => lib}/www/logo.html (100%) rename test/{ => lib}/www/logo.png (100%) rename test/{ => lib}/www/missing-img.html (100%) rename test/{ => lib}/www/navigation/dest.html (100%) rename test/{ => lib}/www/navigation/index.html (100%) rename test/{ => lib}/www/phantomjs.png (100%) rename test/{ => lib}/www/regression/pjs-10690/Windsong.ttf (100%) rename test/{ => lib}/www/regression/pjs-10690/font.css (100%) rename test/{ => lib}/www/regression/pjs-10690/index.html (100%) rename test/{ => lib}/www/regression/pjs-10690/jquery.js (100%) rename test/{ => lib}/www/regression/pjs-13551/child1.html (100%) rename test/{ => lib}/www/regression/pjs-13551/child1a.html (100%) rename test/{ => lib}/www/regression/pjs-13551/child2.html (100%) rename test/{ => lib}/www/regression/pjs-13551/closing-parent.html (100%) rename test/{ => lib}/www/regression/pjs-13551/reloading-parent.html (100%) rename test/{ => lib}/www/regression/webkit-60448.html (100%) rename test/{ => lib}/www/render/image.jpg (100%) rename test/{ => lib}/www/render/index.html (100%) rename test/{ => lib}/www/status.py (100%) rename test/{ => lib}/www/url-encoding.py (100%) rename test/{ => lib}/www/user-agent.html (100%) diff --git a/test/certs/https-snakeoil.crt b/test/lib/certs/https-snakeoil.crt similarity index 100% rename from test/certs/https-snakeoil.crt rename to test/lib/certs/https-snakeoil.crt diff --git a/test/certs/https-snakeoil.key b/test/lib/certs/https-snakeoil.key similarity index 100% rename from test/certs/https-snakeoil.key rename to test/lib/certs/https-snakeoil.key diff --git a/test/fixtures/dummy.js b/test/lib/fixtures/dummy.js similarity index 100% rename from test/fixtures/dummy.js rename to test/lib/fixtures/dummy.js diff --git a/test/fixtures/error-helper.js b/test/lib/fixtures/error-helper.js similarity index 100% rename from test/fixtures/error-helper.js rename to test/lib/fixtures/error-helper.js diff --git a/test/fixtures/parse-error-helper.js b/test/lib/fixtures/parse-error-helper.js similarity index 100% rename from test/fixtures/parse-error-helper.js rename to test/lib/fixtures/parse-error-helper.js diff --git a/test/node_modules/dummy_exposed.js b/test/lib/node_modules/dummy_exposed.js similarity index 100% rename from test/node_modules/dummy_exposed.js rename to test/lib/node_modules/dummy_exposed.js diff --git a/test/node_modules/dummy_file.js b/test/lib/node_modules/dummy_file.js similarity index 100% rename from test/node_modules/dummy_file.js rename to test/lib/node_modules/dummy_file.js diff --git a/test/node_modules/dummy_file2.js b/test/lib/node_modules/dummy_file2.js similarity index 100% rename from test/node_modules/dummy_file2.js rename to test/lib/node_modules/dummy_file2.js diff --git a/test/testharness.js b/test/lib/testharness.js similarity index 99% rename from test/testharness.js rename to test/lib/testharness.js index f68cc6b43..f33822769 100644 --- a/test/testharness.js +++ b/test/lib/testharness.js @@ -1461,6 +1461,13 @@ if (args.test_script === "") { // process_command_line has already issued an error message. phantom.exit(2); } else { + // run-tests.py sets this environment variable to the root of the + // test directory. + expose(sys.env['TEST_DIR'], 'TEST_DIR'); + + // The JS modules in TEST_DIR/lib/node_modules are always available. + require.paths.push(fs.join(sys.env['TEST_DIR'], 'lib', 'node_modules')); + // Reset the library paths for injectJs and require to the // directory containing the test script, so relative imports work // as expected. Unfortunately, phantom.libraryPath is not a diff --git a/test/www/__init__.py b/test/lib/www/__init__.py similarity index 100% rename from test/www/__init__.py rename to test/lib/www/__init__.py diff --git a/test/www/delay.py b/test/lib/www/delay.py similarity index 100% rename from test/www/delay.py rename to test/lib/www/delay.py diff --git a/test/www/echo.py b/test/lib/www/echo.py similarity index 100% rename from test/www/echo.py rename to test/lib/www/echo.py diff --git a/test/www/frameset/frame1-1.html b/test/lib/www/frameset/frame1-1.html similarity index 100% rename from test/www/frameset/frame1-1.html rename to test/lib/www/frameset/frame1-1.html diff --git a/test/www/frameset/frame1-2.html b/test/lib/www/frameset/frame1-2.html similarity index 100% rename from test/www/frameset/frame1-2.html rename to test/lib/www/frameset/frame1-2.html diff --git a/test/www/frameset/frame1.html b/test/lib/www/frameset/frame1.html similarity index 100% rename from test/www/frameset/frame1.html rename to test/lib/www/frameset/frame1.html diff --git a/test/www/frameset/frame2-1.html b/test/lib/www/frameset/frame2-1.html similarity index 100% rename from test/www/frameset/frame2-1.html rename to test/lib/www/frameset/frame2-1.html diff --git a/test/www/frameset/frame2-2.html b/test/lib/www/frameset/frame2-2.html similarity index 100% rename from test/www/frameset/frame2-2.html rename to test/lib/www/frameset/frame2-2.html diff --git a/test/www/frameset/frame2-3.html b/test/lib/www/frameset/frame2-3.html similarity index 100% rename from test/www/frameset/frame2-3.html rename to test/lib/www/frameset/frame2-3.html diff --git a/test/www/frameset/frame2.html b/test/lib/www/frameset/frame2.html similarity index 100% rename from test/www/frameset/frame2.html rename to test/lib/www/frameset/frame2.html diff --git a/test/www/frameset/index.html b/test/lib/www/frameset/index.html similarity index 100% rename from test/www/frameset/index.html rename to test/lib/www/frameset/index.html diff --git a/test/www/hello.html b/test/lib/www/hello.html similarity index 100% rename from test/www/hello.html rename to test/lib/www/hello.html diff --git a/test/www/iframe.html b/test/lib/www/iframe.html similarity index 100% rename from test/www/iframe.html rename to test/lib/www/iframe.html diff --git a/test/www/includejs.js b/test/lib/www/includejs.js similarity index 100% rename from test/www/includejs.js rename to test/lib/www/includejs.js diff --git a/test/www/includejs1.html b/test/lib/www/includejs1.html similarity index 100% rename from test/www/includejs1.html rename to test/lib/www/includejs1.html diff --git a/test/www/includejs2.html b/test/lib/www/includejs2.html similarity index 100% rename from test/www/includejs2.html rename to test/lib/www/includejs2.html diff --git a/test/www/js-infinite-loop.html b/test/lib/www/js-infinite-loop.html similarity index 100% rename from test/www/js-infinite-loop.html rename to test/lib/www/js-infinite-loop.html diff --git a/test/www/logo.html b/test/lib/www/logo.html similarity index 100% rename from test/www/logo.html rename to test/lib/www/logo.html diff --git a/test/www/logo.png b/test/lib/www/logo.png similarity index 100% rename from test/www/logo.png rename to test/lib/www/logo.png diff --git a/test/www/missing-img.html b/test/lib/www/missing-img.html similarity index 100% rename from test/www/missing-img.html rename to test/lib/www/missing-img.html diff --git a/test/www/navigation/dest.html b/test/lib/www/navigation/dest.html similarity index 100% rename from test/www/navigation/dest.html rename to test/lib/www/navigation/dest.html diff --git a/test/www/navigation/index.html b/test/lib/www/navigation/index.html similarity index 100% rename from test/www/navigation/index.html rename to test/lib/www/navigation/index.html diff --git a/test/www/phantomjs.png b/test/lib/www/phantomjs.png similarity index 100% rename from test/www/phantomjs.png rename to test/lib/www/phantomjs.png diff --git a/test/www/regression/pjs-10690/Windsong.ttf b/test/lib/www/regression/pjs-10690/Windsong.ttf similarity index 100% rename from test/www/regression/pjs-10690/Windsong.ttf rename to test/lib/www/regression/pjs-10690/Windsong.ttf diff --git a/test/www/regression/pjs-10690/font.css b/test/lib/www/regression/pjs-10690/font.css similarity index 100% rename from test/www/regression/pjs-10690/font.css rename to test/lib/www/regression/pjs-10690/font.css diff --git a/test/www/regression/pjs-10690/index.html b/test/lib/www/regression/pjs-10690/index.html similarity index 100% rename from test/www/regression/pjs-10690/index.html rename to test/lib/www/regression/pjs-10690/index.html diff --git a/test/www/regression/pjs-10690/jquery.js b/test/lib/www/regression/pjs-10690/jquery.js similarity index 100% rename from test/www/regression/pjs-10690/jquery.js rename to test/lib/www/regression/pjs-10690/jquery.js diff --git a/test/www/regression/pjs-13551/child1.html b/test/lib/www/regression/pjs-13551/child1.html similarity index 100% rename from test/www/regression/pjs-13551/child1.html rename to test/lib/www/regression/pjs-13551/child1.html diff --git a/test/www/regression/pjs-13551/child1a.html b/test/lib/www/regression/pjs-13551/child1a.html similarity index 100% rename from test/www/regression/pjs-13551/child1a.html rename to test/lib/www/regression/pjs-13551/child1a.html diff --git a/test/www/regression/pjs-13551/child2.html b/test/lib/www/regression/pjs-13551/child2.html similarity index 100% rename from test/www/regression/pjs-13551/child2.html rename to test/lib/www/regression/pjs-13551/child2.html diff --git a/test/www/regression/pjs-13551/closing-parent.html b/test/lib/www/regression/pjs-13551/closing-parent.html similarity index 100% rename from test/www/regression/pjs-13551/closing-parent.html rename to test/lib/www/regression/pjs-13551/closing-parent.html diff --git a/test/www/regression/pjs-13551/reloading-parent.html b/test/lib/www/regression/pjs-13551/reloading-parent.html similarity index 100% rename from test/www/regression/pjs-13551/reloading-parent.html rename to test/lib/www/regression/pjs-13551/reloading-parent.html diff --git a/test/www/regression/webkit-60448.html b/test/lib/www/regression/webkit-60448.html similarity index 100% rename from test/www/regression/webkit-60448.html rename to test/lib/www/regression/webkit-60448.html diff --git a/test/www/render/image.jpg b/test/lib/www/render/image.jpg similarity index 100% rename from test/www/render/image.jpg rename to test/lib/www/render/image.jpg diff --git a/test/www/render/index.html b/test/lib/www/render/index.html similarity index 100% rename from test/www/render/index.html rename to test/lib/www/render/index.html diff --git a/test/www/status.py b/test/lib/www/status.py similarity index 100% rename from test/www/status.py rename to test/lib/www/status.py diff --git a/test/www/url-encoding.py b/test/lib/www/url-encoding.py similarity index 100% rename from test/www/url-encoding.py rename to test/lib/www/url-encoding.py diff --git a/test/www/user-agent.html b/test/lib/www/user-agent.html similarity index 100% rename from test/www/user-agent.html rename to test/lib/www/user-agent.html diff --git a/test/module/webpage/capture-content.js b/test/module/webpage/capture-content.js index fa2e44efc..dfee09052 100644 --- a/test/module/webpage/capture-content.js +++ b/test/module/webpage/capture-content.js @@ -1,9 +1,7 @@ var content; setup(function () { var fs = require('fs'); - // libraryPath is test/module/webpage - content = fs.read(fs.join(phantom.libraryPath, - "../../www/hello.html")); + content = fs.read(fs.join(TEST_DIR, "lib/www/hello.html")); }); // XFAIL: This feature had to be backed out for breaking WebSockets. diff --git a/test/module/webpage/on-error.js b/test/module/webpage/on-error.js index 683c061fb..b5853613b 100644 --- a/test/module/webpage/on-error.js +++ b/test/module/webpage/on-error.js @@ -1,4 +1,5 @@ var webpage = require('webpage'); +var fs = require('fs'); test(function () { var page = webpage.create(); @@ -66,6 +67,9 @@ test(function () { assert_is_true(page.evaluate(function() { return window.caughtError; })); }, "should not report errors that were caught"); +var helperBase = "error-helper.js"; +var helperFile = fs.join(TEST_DIR, "lib", "fixtures", helperBase); + function check_stack(message, stack) { assert_equals(message, "ReferenceError: Can't find variable: referenceError"); @@ -85,11 +89,8 @@ function check_stack(message, stack) { } } -var helperBase = "error-helper.js"; -var helperFile = "../../fixtures/" + helperBase; -assert_is_true(phantom.injectJs(helperFile)); - test(function () { + assert_is_true(phantom.injectJs(helperFile)); try { ErrorHelper.foo(); } catch (e) { @@ -99,7 +100,6 @@ test(function () { async_test(function () { var page = webpage.create(); - page.libraryPath = phantom.libraryPath; assert_is_true(page.injectJs(helperFile)); page.onError = this.step_func_done(check_stack); diff --git a/test/regression/pjs-13551.js b/test/regression/pjs-13551.js index 844e219d7..60695dfab 100644 --- a/test/regression/pjs-13551.js +++ b/test/regression/pjs-13551.js @@ -6,7 +6,7 @@ var webpage = require('webpage'); function test_template(parent, action) { var page; var url = TEST_HTTP_BASE + - "/regression/pjs-13551/" + parent + "-parent.html"; + "regression/pjs-13551/" + parent + "-parent.html"; var s_callback0, s_callback1, s_callback2; function callback0 (n) { diff --git a/test/run-tests.py b/test/run-tests.py index 060d53f03..253959689 100755 --- a/test/run-tests.py +++ b/test/run-tests.py @@ -109,8 +109,8 @@ CIPHERLIST_2_7_9 = ( '!eNULL:!MD5:!DSS:!RC4' ) def wrap_socket_ssl(sock, base_path): - crtfile = os.path.join(base_path, 'certs/https-snakeoil.crt') - keyfile = os.path.join(base_path, 'certs/https-snakeoil.key') + crtfile = os.path.join(base_path, 'lib/certs/https-snakeoil.crt') + keyfile = os.path.join(base_path, 'lib/certs/https-snakeoil.key') try: ctx = ssl.create_default_context(ssl.Purpose.CLIENT_AUTH) @@ -367,7 +367,7 @@ class HTTPTestServer(object): self.httpd = None self.httpsd = None self.base_path = base_path - self.www_path = os.path.join(base_path, 'www') + self.www_path = os.path.join(base_path, 'lib/www') self.signal_error = signal_error self.verbose = verbose @@ -669,6 +669,13 @@ class TAPTestGroup(TestGroup): self.add_skip(out[(i+1):], "All further output ignored") return + if any(msg.startswith("ERROR:") for msg in messages): + self.add_error(messages, "Before tests") + messages = [] + elif messages: + self.add_error(messages, "Stray diagnostic") + messages = [] + prev_point = 0 for i in range(i+1, len(out)): @@ -743,8 +750,8 @@ class TAPTestGroup(TestGroup): class TestRunner(object): def __init__(self, base_path, phantomjs_exe, options): self.base_path = base_path - self.cert_path = os.path.join(base_path, 'certs') - self.harness = os.path.join(base_path, 'testharness.js') + self.cert_path = os.path.join(base_path, 'lib/certs') + self.harness = os.path.join(base_path, 'lib/testharness.js') self.phantomjs_exe = phantomjs_exe self.verbose = options.verbose self.debugger = options.debugger @@ -960,6 +967,8 @@ class TestRunner(object): def init(): base_path = os.path.normpath(os.path.dirname(os.path.abspath(__file__))) + os.environ["TEST_DIR"] = base_path + phantomjs_exe = os.path.normpath(base_path + '/../bin/phantomjs') if sys.platform in ('win32', 'cygwin'): phantomjs_exe += '.exe' @@ -1009,6 +1018,15 @@ def init(): # usually written, e.g. UTC+1 would be xxx-1:00. os.environ["TZ"] = "CIST-12:45:00" + # Run all the tests in the "C" locale. (A UTF-8-based locale + # which is thoroughly different from "C" would flush out more + # bugs, but we have no way of knowing if such a locale exists.) + for var in list(os.environ.keys()): + if var[:3] == 'LC_' or var[:4] == 'LANG': + del os.environ[var] + + os.environ["LANG"] = "C" + return runner def main(): diff --git a/test/writing-tests.md b/test/writing-tests.md index 1c827fbb8..58c2fea3c 100644 --- a/test/writing-tests.md +++ b/test/writing-tests.md @@ -8,11 +8,11 @@ of test subdirectories is in [the `TESTS` variable in `run-tests.py`](run-tests.py#L26).) In addition to all of the usual PhantomJS API, these scripts have -access to a special testing API, loosely based on -[W3C testharness.js](https://github.com/w3c/testharness.js) and -defined in [`testharness.js`](testharness.js) in this directory. They -also have access to HTTP and HTTPS servers on `localhost`, which serve -the files in the [`www`](www) directory. +access to a special testing API, loosely based on [W3C +testharness.js](https://github.com/w3c/testharness.js) and defined +in [`lib/testharness.js`](lib/testharness.js). They also have +access to HTTP and HTTPS servers on `localhost`, which serve the +files in the [`lib/www`](lib/www) directory. ## The Structure of Test Scripts @@ -561,20 +561,20 @@ respective streams. ## Test Server Modules -The HTTP and HTTPS servers exposed to the test suite serve the static -files in the `www` subdirectory with URLs corresponding to their paths -relative to that directory. If you need more complicated server -behavior than that, you can write custom Python code that executes -when the server receives a request. Any `.py` file below the `www` -directory will be invoked to provide the response for that path -*without* the `.py` suffix. (For instance, `www/echo.py` provides -responses for `TEST_HTTP_BASE + 'echo'`.) Such files must define a -top-level function named `handle_request`. This function receives a -single argument, which is an instance of a subclass of +The HTTP and HTTPS servers exposed to the test suite serve the +static files in the `lib/www` subdirectory, with URLs corresponding +to their paths relative to that directory. If you need more +complicated server behavior than that, you can write custom Python +code that executes when the server receives a request. Any `.py` +file below `lib/www` will be invoked to provide the response for +that path *without* the `.py` suffix. (For instance, `lib/www/echo.py` +provides responses for `TEST_HTTP_BASE + 'echo'`.) Such files must +define a top-level function named `handle_request`. This function +receives a single argument, which is an instance of a subclass of [`BaseHTTPServer.BaseHTTPRequestHandler`](https://docs.python.org/2/library/basehttpserver.html#BaseHTTPServer.BaseHTTPRequestHandler). The request headers and body (if any) may be retrieved from this -object. The function must use the `send_response`, `send_header`, and -`end_headers` methods of this object to generate HTTP response +object. The function must use the `send_response`, `send_header`, +and `end_headers` methods of this object to generate HTTP response headers, and then return a *file-like object* (**not** a string) containing the response body. The function is responsible for generating appropriate `Content-Type` and `Content-Length` headers;