microsoft/pyright
1
1
Fork 0
mirror of https://github.com/microsoft/pyright.git synced 2024-07-14 19:10:39 +03:00
Code Issues Projects Releases Wiki Activity

Mass reformat ts/js/md with ESLint quick fixes and prettier (#113)

Browse Source
This commit is contained in:
Jake Bailey 2020-02-24 16:52:23 -08:00 committed by GitHub
parent e17977c928
commit 8d2cca7289
143 changed files with 11201 additions and 6061 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
.eslintignore
View File

@ -1 +1,4 @@
**/dist/**
**/out/**
**/node_modules/**
**/tests/fourslash/** **/tests/fourslash/**

1
.eslintrc.json
View File

@ -1,6 +1,7 @@
{ {
"extends": [ "extends": [
"eslint:recommended", "eslint:recommended",
"prettier",
"plugin:@typescript-eslint/recommended" "plugin:@typescript-eslint/recommended"
], ],
"env": { "env": {

6
.prettierignore Normal file
View File

@ -0,0 +1,6 @@
*.json
**/dist/**
**/out/**
**/client/server/**
**/typeshed-fallback/**
**/.github/**

6
.prettierrc Normal file
View File

@ -0,0 +1,6 @@
{
"singleQuote": true,
"tabWidth": 4,
"useTabs": false,
"printWidth": 120
}

6
.vscode/extensions.json vendored Normal file
View File

@ -0,0 +1,6 @@
{
"recommendations": [
"dbaeumer.vscode-eslint",
"esbenp.prettier-vscode"
]
}

17
.vscode/settings.json vendored Normal file
View File

@ -0,0 +1,17 @@
{
"[typescript]": {
"editor.defaultFormatter": "esbenp.prettier-vscode",
"editor.formatOnSave": true
},
"[javascript]": {
"editor.defaultFormatter": "esbenp.prettier-vscode",
"editor.formatOnSave": true
},
"[markdown]": {
"editor.defaultFormatter": "esbenp.prettier-vscode",
"editor.formatOnSave": true
},
"editor.codeActionsOnSave": {
"source.fixAll.eslint": true
}
}

79
README.md
View File

@ -3,53 +3,63 @@
# Static type checker for Python # Static type checker for Python
### Speed ### Speed
Pyright is a fast type checker meant for large Python source bases. It can run in a “watch” mode and performs fast incremental updates when files are modified.
Pyright is a fast type checker meant for large Python source bases. It can run in a "watch" mode and performs fast incremental updates when files are modified.
### Configurability ### Configurability
Pyright supports [configuration files](/docs/configuration.md) that provide granular control over settings. Different “execution environments” can be associated with subdirectories within a source base. Each environment can specify different module search paths, python language versions, and platform targets.
Pyright supports [configuration files](/docs/configuration.md) that provide granular control over settings. Different "execution environments" can be associated with subdirectories within a source base. Each environment can specify different module search paths, python language versions, and platform targets.
### Type Checking Features ### Type Checking Features
* [PEP 484](https://www.python.org/dev/peps/pep-0484/) type hints including generics
* [PEP 526](https://www.python.org/dev/peps/pep-0526/) syntax for variable annotations - [PEP 484](https://www.python.org/dev/peps/pep-0484/) type hints including generics
* [PEP 544](https://www.python.org/dev/peps/pep-0544/) structural subtyping - [PEP 526](https://www.python.org/dev/peps/pep-0526/) syntax for variable annotations
* [PEP 589](https://www.python.org/dev/peps/pep-0589/) typed dictionaries - [PEP 544](https://www.python.org/dev/peps/pep-0544/) structural subtyping
* Type inference for function return values, instance variables, class variables, and globals - [PEP 589](https://www.python.org/dev/peps/pep-0589/) typed dictionaries
* Smart type constraints that understand conditional code flow constructs like if/else statements - Type inference for function return values, instance variables, class variables, and globals
- Smart type constraints that understand conditional code flow constructs like if/else statements
### VS Code Integration ### VS Code Integration
Pyright ships as both a command-line tool and a VS Code extension that provides many powerful features that help improve programming efficiency. Pyright ships as both a command-line tool and a VS Code extension that provides many powerful features that help improve programming efficiency.
### VS Code Language Features ### VS Code Language Features
The VS Code extension supports many time-saving language features including: The VS Code extension supports many time-saving language features including:
* Intelligent type completion of keywords, symbols, and import names appears when editing - Intelligent type completion of keywords, symbols, and import names appears when editing
* Import statements are automatically inserted when necessary for type completions - Import statements are automatically inserted when necessary for type completions
* Signature completion tips help when filling in arguments for a call - Signature completion tips help when filling in arguments for a call
* Hover over symbols to provide type information and doc strings - Hover over symbols to provide type information and doc strings
* Find Definitions to quickly go to the location of a symbol’s definition - Find Definitions to quickly go to the location of a symbol's definition
* Find References to find all references to a symbol within a code base - Find References to find all references to a symbol within a code base
* Rename Symbol to rename all references to a symbol within a code base - Rename Symbol to rename all references to a symbol within a code base
* Find Symbols within the current document or within the entire workspace - Find Symbols within the current document or within the entire workspace
* Organize Imports command for automatically ordering imports according to PEP8 rules - Organize Imports command for automatically ordering imports according to PEP8 rules
* Type stub generation for third-party libraries - Type stub generation for third-party libraries
### Built-in Type Stubs ### Built-in Type Stubs
Pyright includes a recent copy of the stdlib type stubs from [Typeshed](https://github.com/python/typeshed). It can be configured to use another (perhaps more recent or modified) copy of the Typeshed type stubs. Of course, it also works with custom type stub files that are part of your project. Pyright includes a recent copy of the stdlib type stubs from [Typeshed](https://github.com/python/typeshed). It can be configured to use another (perhaps more recent or modified) copy of the Typeshed type stubs. Of course, it also works with custom type stub files that are part of your project.
### Command-line Tool or Visual Studio Code Extension ### Command-line Tool or Visual Studio Code Extension
Pyright includes both a [command-line tool](/docs/command-line.md) and an [extension for Visual Studio Code](https://marketplace.visualstudio.com/items?itemName=ms-pyright.pyright) that implements the [Language Server Protocol](https://microsoft.github.io/language-server-protocol/). Pyright includes both a [command-line tool](/docs/command-line.md) and an [extension for Visual Studio Code](https://marketplace.visualstudio.com/items?itemName=ms-pyright.pyright) that implements the [Language Server Protocol](https://microsoft.github.io/language-server-protocol/).
For rich Python editing and debugging capabilities with Visual Studio Code, be sure to also install the official [Microsoft Python extension for Visual Studio Code](https://marketplace.visualstudio.com/itemdetails?itemName=ms-python.python) as Pyright only provides syntax and type checking. For rich Python editing and debugging capabilities with Visual Studio Code, be sure to also install the official [Microsoft Python extension for Visual Studio Code](https://marketplace.visualstudio.com/itemdetails?itemName=ms-python.python) as Pyright only provides syntax and type checking.
## Installation ## Installation
### VS Code Extension ### VS Code Extension
You can install the latest-published version of the Pyright VS Code extension directly from VS Code. Simply open the extensions panel and search for `pyright`. You can install the latest-published version of the Pyright VS Code extension directly from VS Code. Simply open the extensions panel and search for `pyright`.
### Vim ### Vim
For vim/neovim users, you can install [coc-pyright](https://github.com/fannheyward/coc-pyright), Pyright extension for coc.nvim. For vim/neovim users, you can install [coc-pyright](https://github.com/fannheyward/coc-pyright), Pyright extension for coc.nvim.
### Command-line ### Command-line
The latest version of the command-line tool can be installed with npm, which is part of node. If you don't have a recent version of node on your system, install that first from [nodejs.org](nodejs.org). The latest version of the command-line tool can be installed with npm, which is part of node. If you don't have a recent version of node on your system, install that first from [nodejs.org](nodejs.org).
To install pyright globally: To install pyright globally:
@ -64,42 +74,41 @@ Once installed, you can run the tool from the command line as follows:
To update to the latest version: To update to the latest version:
`sudo npm update -g pyright` `sudo npm update -g pyright`
## Using Pyright with VS Code Python Extension ## Using Pyright with VS Code Python Extension
Pyright provides some features that overlap with functionality provided by the standard VS Code Python extension: “hover”, type completion, definitions, references, rename symbols, etc. You may see duplicate results if Pyright is installed alongside the Python extension. There is currently no way to disable this functionality in the Python extension. If you want to disable these features in Pyright, there is a setting to do so: `pyright.disableLanguageServices`.
Pyright provides some features that overlap with functionality provided by the standard VS Code Python extension: "hover", type completion, definitions, references, rename symbols, etc. You may see duplicate results if Pyright is installed alongside the Python extension. There is currently no way to disable this functionality in the Python extension. If you want to disable these features in Pyright, there is a setting to do so: `pyright.disableLanguageServices`.
## Documentation ## Documentation
* [Getting Started with Type Checking](/docs/getting-started.md)
* [Command-line Options](/docs/command-line.md)
* [Configuration](/docs/configuration.md)
* [Settings](/docs/settings.md)
* [Comments](/docs/comments.md)
* [Import Resolution](/docs/import-resolution.md)
* [Type Stubs](/docs/type-stubs.md)
* [Commands](/docs/commands.md)
* [Building & Debugging](/docs/build-debug.md)
* [Pyright Internals](/docs/internals.md)
- [Getting Started with Type Checking](/docs/getting-started.md)
- [Command-line Options](/docs/command-line.md)
- [Configuration](/docs/configuration.md)
- [Settings](/docs/settings.md)
- [Comments](/docs/comments.md)
- [Import Resolution](/docs/import-resolution.md)
- [Type Stubs](/docs/type-stubs.md)
- [Commands](/docs/commands.md)
- [Building & Debugging](/docs/build-debug.md)
- [Pyright Internals](/docs/internals.md)
## Limitations ## Limitations
Pyright currently provides support for Python 3.0 and newer. There is currently no plan to support older versions. Pyright currently provides support for Python 3.0 and newer. There is currently no plan to support older versions.
## Community ## Community
Do you have questions about Pyright or Python type annotations in general? Post your questions in this [gitter channel](https://gitter.im/microsoft-pyright/community). Do you have questions about Pyright or Python type annotations in general? Post your questions in this [gitter channel](https://gitter.im/microsoft-pyright/community).
## FAQ ## FAQ
**Q:** What is the difference between pyright and the [Microsoft Python Visual Studio Code plugin](https://github.com/Microsoft/vscode-python)? **Q:** What is the difference between pyright and the [Microsoft Python Visual Studio Code plugin](https://github.com/Microsoft/vscode-python)?
**A:** Pyright is focused on type checking. The Python VS Code plugin is Microsoft’s officially-supported extension for VS Code and provides a diverse array of features including debugging, test case management, linter plugins, and more. Pyright can be used alongside the Microsoft Python extension. **A:** Pyright is focused on type checking. The Python VS Code plugin is Microsoft's officially-supported extension for VS Code and provides a diverse array of features including debugging, test case management, linter plugins, and more. Pyright can be used alongside the Microsoft Python extension.
**Q:** What is the long-term plan for Pyright? **Q:** What is the long-term plan for Pyright?
**A:** Pyright is a side project with no dedicated team. There is no guarantee of continued development on the project. If you find it useful, feel free to use it and contribute to the code base. **A:** Pyright is a side project with no dedicated team. There is no guarantee of continued development on the project. If you find it useful, feel free to use it and contribute to the code base.
## Contributing ## Contributing
This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.microsoft.com. This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.microsoft.com.

42
client/README.md
View File

@ -1,35 +1,39 @@
# Static type checker for Python # Static type checker for Python
### Speed ### Speed
Pyright is a fast type checker meant for large Python source bases. It can run in a “watch” mode and performs fast incremental updates when files are modified.
Pyright is a fast type checker meant for large Python source bases. It can run in a "watch" mode and performs fast incremental updates when files are modified.
### Configurability ### Configurability
Pyright supports [configuration files](/docs/configuration.md) that provide granular control over settings. Different “execution environments” can be associated with subdirectories within a source base. Each environment can specify different module search paths, python language versions, and platform targets.
Pyright supports [configuration files](/docs/configuration.md) that provide granular control over settings. Different "execution environments" can be associated with subdirectories within a source base. Each environment can specify different module search paths, python language versions, and platform targets.
### Type Checking Features ### Type Checking Features
* [PEP 484](https://www.python.org/dev/peps/pep-0484/) type hints including generics
* [PEP 526](https://www.python.org/dev/peps/pep-0526/) syntax for variable annotations - [PEP 484](https://www.python.org/dev/peps/pep-0484/) type hints including generics
* [PEP 544](https://www.python.org/dev/peps/pep-0544/) structural subtyping - [PEP 526](https://www.python.org/dev/peps/pep-0526/) syntax for variable annotations
* [PEP 589](https://www.python.org/dev/peps/pep-0589/) typed dictionaries - [PEP 544](https://www.python.org/dev/peps/pep-0544/) structural subtyping
* Type inference for function return values, instance variables, class variables, and globals - [PEP 589](https://www.python.org/dev/peps/pep-0589/) typed dictionaries
* Smart type constraints that understand conditional code flow constructs like if/else statements - Type inference for function return values, instance variables, class variables, and globals
- Smart type constraints that understand conditional code flow constructs like if/else statements
### VS Code Language Features ### VS Code Language Features
The VS Code extension supports many time-saving language features including: The VS Code extension supports many time-saving language features including:
* Intelligent type completion of keywords, symbols, and import names appears when editing - Intelligent type completion of keywords, symbols, and import names appears when editing
* Import statements are automatically inserted when necessary for type completions - Import statements are automatically inserted when necessary for type completions
* Signature completion tips help when filling in arguments for a call - Signature completion tips help when filling in arguments for a call
* Hover over symbols to provide type information and doc strings - Hover over symbols to provide type information and doc strings
* Find Definitions to quickly go to the location of a symbol’s definition - Find Definitions to quickly go to the location of a symbol's definition
* Find References to find all references to a symbol within a code base - Find References to find all references to a symbol within a code base
* Rename Symbol to rename all references to a symbol within a code base - Rename Symbol to rename all references to a symbol within a code base
* Find Symbols within the current document or within the entire workspace - Find Symbols within the current document or within the entire workspace
* Organize Imports command for automatically ordering imports according to PEP8 rules - Organize Imports command for automatically ordering imports according to PEP8 rules
* Type stub generation for third-party libraries - Type stub generation for third-party libraries
### Built-in Type Stubs ### Built-in Type Stubs
Pyright includes a recent copy of the stdlib type stubs from [Typeshed](https://github.com/python/typeshed). It can be configured to use another (perhaps more recent or modified) copy of the Typeshed type stubs. Of course, it also works with custom type stub files that are part of your project. Pyright includes a recent copy of the stdlib type stubs from [Typeshed](https://github.com/python/typeshed). It can be configured to use another (perhaps more recent or modified) copy of the Typeshed type stubs. Of course, it also works with custom type stub files that are part of your project.
For more details, refer to the [README](https://github.com/Microsoft/pyright/blob/master/README.md) on the Pyright GitHub site. For more details, refer to the [README](https://github.com/Microsoft/pyright/blob/master/README.md) on the Pyright GitHub site.

46
client/package-lock.json generated
View File

@ -5,9 +5,9 @@
"requires": true, "requires": true,
"dependencies": { "dependencies": {
"@types/node": { "@types/node": {
"version": "12.12.14", "version": "13.7.4",
"resolved": "https://registry.npmjs.org/@types/node/-/node-12.12.14.tgz", "resolved": "https://registry.npmjs.org/@types/node/-/node-13.7.4.tgz",
"integrity": "sha512-u/SJDyXwuihpwjXy7hOOghagLEV1KdAST6syfnOk6QZAMzZuWZqXy5aYYZbh8Jdpd4escVFP0MvftHNDb9pruA==", "integrity": "sha512-oVeL12C6gQS/GAExndigSaLxTrKpQPxewx9bOcwfvJiJge4rr7wNaph4J+ns5hrmIV2as5qxqN8YKthn9qh0jw==",
"dev": true "dev": true
}, },
"agent-base": { "agent-base": {
@ -838,9 +838,9 @@
} }
}, },
"readable-stream": { "readable-stream": {
"version": "3.4.0", "version": "3.6.0",
"resolved": "https://registry.npmjs.org/readable-stream/-/readable-stream-3.4.0.tgz", "resolved": "https://registry.npmjs.org/readable-stream/-/readable-stream-3.6.0.tgz",
"integrity": "sha512-jItXPLmrSR8jmTRmRWJXCnGJsfy85mB3Wd/uINMXA65yrnFo0cPClFIUWzo2najVNSl+mx7/4W8ttlLWJe99pQ==", "integrity": "sha512-BViHy7LKeTz4oNnkcLJ+lVSL6vpiFeX6/d3oSH8zCW7UxP2onchk+vTGB143xuFjHS3deTgkKoXXymXqymiIdA==",
"dev": true, "dev": true,
"requires": { "requires": {
"inherits": "^2.0.3", "inherits": "^2.0.3",
@ -1016,9 +1016,9 @@
} }
}, },
"typescript": { "typescript": {
"version": "3.7.3", "version": "3.8.2",
"resolved": "https://registry.npmjs.org/typescript/-/typescript-3.7.3.tgz", "resolved": "https://registry.npmjs.org/typescript/-/typescript-3.8.2.tgz",
"integrity": "sha512-Mcr/Qk7hXqFBXMN7p7Lusj1ktCBydylfQM/FZCk5glCNQJrCUKPkMHdo9R0MTFWsC/4kPFvDS0fDPvukfCkFsw==", "integrity": "sha512-EgOVgL/4xfVrCMbhYKUQTdF37SQn4Iw73H5BgCrF1Abdun7Kwy/QZsE/ssAy0y4LxBbvua3PIbFsbRczWWnDdQ==",
"dev": true "dev": true
}, },
"uc.micro": { "uc.micro": {
@ -1082,9 +1082,9 @@
} }
}, },
"vsce": { "vsce": {
"version": "1.70.0", "version": "1.73.0",
"resolved": "https://registry.npmjs.org/vsce/-/vsce-1.70.0.tgz", "resolved": "https://registry.npmjs.org/vsce/-/vsce-1.73.0.tgz",
"integrity": "sha512-mBTbVrWL348jODwfmaR+yXrlzb8EABGCT067C4shKOXriWiuMQi4/uCbFm6TUBcfnzTYLJv+DKa0VnKU8yEAjA==", "integrity": "sha512-6W37Ebbkj3uF3WhT+SCfRtsneRQEFcGvf/XYz+b6OAgDCj4gPurWyDVrqw/HLsbP1WflGIyUfVZ8t5M7kQp6Uw==",
"dev": true, "dev": true,
"requires": { "requires": {
"azure-devops-node-api": "^7.2.0", "azure-devops-node-api": "^7.2.0",
@ -1125,9 +1125,9 @@
} }
}, },
"vscode-jsonrpc": { "vscode-jsonrpc": {
"version": "5.0.0-next.2", "version": "5.0.1",
"resolved": "https://registry.npmjs.org/vscode-jsonrpc/-/vscode-jsonrpc-5.0.0-next.2.tgz", "resolved": "https://registry.npmjs.org/vscode-jsonrpc/-/vscode-jsonrpc-5.0.1.tgz",
"integrity": "sha512-Q3/jabZUNviCG9hhF6hHWjhrABevPF9mv0aiE2j8BYCAP2k+aHTpjMyk+04MzaAqWYwXdQuZkLSbcYCCqbzJLg==" "integrity": "sha512-JvONPptw3GAQGXlVV2utDcHx0BiY34FupW/kI6mZ5x06ER5DdPG/tXWMVHjTNULF5uKPOUUD0SaXg5QaubJL0A=="
}, },
"vscode-languageclient": { "vscode-languageclient": {
"version": "5.3.0-next.9", "version": "5.3.0-next.9",
@ -1155,18 +1155,18 @@
} }
}, },
"vscode-languageserver-protocol": { "vscode-languageserver-protocol": {
"version": "3.15.0-next.9", "version": "3.15.3",
"resolved": "https://registry.npmjs.org/vscode-languageserver-protocol/-/vscode-languageserver-protocol-3.15.0-next.9.tgz", "resolved": "https://registry.npmjs.org/vscode-languageserver-protocol/-/vscode-languageserver-protocol-3.15.3.tgz",
"integrity": "sha512-b9PAxouMmtsLEe8ZjbIMPb7wRWPhckGfgjwZLmp/dWnaAuRPYtY3lGO0/rNbLc3jKIqCVlnEyYVFKalzDAzj0g==", "integrity": "sha512-zrMuwHOAQRhjDSnflWdJG+O2ztMWss8GqUUB8dXLR/FPenwkiBNkMIJJYfSN6sgskvsF0rHAoBowNQfbyZnnvw==",
"requires": { "requires": {
"vscode-jsonrpc": "^5.0.0-next.2", "vscode-jsonrpc": "^5.0.1",
"vscode-languageserver-types": "^3.15.0-next.5" "vscode-languageserver-types": "3.15.1"
} }
}, },
"vscode-languageserver-types": { "vscode-languageserver-types": {
"version": "3.15.0-next.5", "version": "3.15.1",
"resolved": "https://registry.npmjs.org/vscode-languageserver-types/-/vscode-languageserver-types-3.15.0-next.5.tgz", "resolved": "https://registry.npmjs.org/vscode-languageserver-types/-/vscode-languageserver-types-3.15.1.tgz",
"integrity": "sha512-7hrELhTeWieUgex3+6692KjCkcmO/+V/bFItM5MHGcBotzwmjEuXjapLLYTYhIspuJ1ibRSik5MhX5YwLpsPiw==" "integrity": "sha512-+a9MPUQrNGRrGU630OGbYVQ+11iOIovjCkqxajPa9w57Sd5ruK8WQNsslzpa0x/QJqC8kRc2DUxWjIFwoNm4ZQ=="
}, },
"vscode-test": { "vscode-test": {
"version": "0.4.3", "version": "0.4.3",

4
client/package.json
View File

@ -116,8 +116,8 @@
"postinstall": "node ./node_modules/vscode/bin/install" "postinstall": "node ./node_modules/vscode/bin/install"
}, },
"devDependencies": { "devDependencies": {
"typescript": "^3.7.3", "typescript": "^3.8.2",
"vsce": "^1.70.0", "vsce": "^1.73.0",
"vscode": "^1.1.36" "vscode": "^1.1.36"
}, },
"dependencies": { "dependencies": {

15
client/schemas/pyrightconfig.schema.json
View File

@ -5,8 +5,17 @@
"definitions": { "definitions": {
"diagnostic": { "diagnostic": {
"anyOf": [ "anyOf": [
{ "type": "boolean" }, {
{ "type": "string", "enum": [ "none", "warning", "error" ] } "type": "boolean"
},
{
"type": "string",
"enum": [
"none",
"warning",
"error"
]
}
] ]
} }
}, },
@ -93,7 +102,7 @@
"enableTypeIgnoreComments": { "enableTypeIgnoreComments": {
"$id": "#/properties/enableTypeIgnoreComments", "$id": "#/properties/enableTypeIgnoreComments",
"type": "boolean", "type": "boolean",
"title": "Allow “# type: ignore” comments", "title": "Allow \"# type: ignore\" comments",
"default": "true" "default": "true"
}, },
"reportTypeshedErrors": { "reportTypeshedErrors": {

43
client/src/extension.ts
View File

@ -20,7 +20,7 @@ import { ProgressReporting } from './progress';
export function activate(context: ExtensionContext) { export function activate(context: ExtensionContext) {
const bundlePath = context.asAbsolutePath(path.join('server', 'server.bundle.js')); const bundlePath = context.asAbsolutePath(path.join('server', 'server.bundle.js'));
const nonBundlePath = context.asAbsolutePath(path.join('server', 'src', 'server.js')); const nonBundlePath = context.asAbsolutePath(path.join('server', 'src', 'server.js'));
const debugOptions = { execArgv: ["--nolazy", "--inspect=6600"] }; const debugOptions = { execArgv: ['--nolazy', '--inspect=6600'] };
// If the extension is launched in debug mode, then the debug server options are used. // If the extension is launched in debug mode, then the debug server options are used.
const serverOptions: ServerOptions = { const serverOptions: ServerOptions = {
@ -28,22 +28,27 @@ export function activate(context: ExtensionContext) {
// In debug mode, use the non-bundled code if it's present. The production // In debug mode, use the non-bundled code if it's present. The production
// build includes only the bundled package, so we don't want to crash if // build includes only the bundled package, so we don't want to crash if
// someone starts the production extension in debug mode. // someone starts the production extension in debug mode.
debug: { module: fs.existsSync(nonBundlePath) ? nonBundlePath : bundlePath, debug: {
transport: TransportKind.ipc, options: debugOptions } module: fs.existsSync(nonBundlePath) ? nonBundlePath : bundlePath,
transport: TransportKind.ipc,
options: debugOptions
} }
};
// Options to control the language client // Options to control the language client
const clientOptions: LanguageClientOptions = { const clientOptions: LanguageClientOptions = {
// Register the server for python source files. // Register the server for python source files.
documentSelector: [{ documentSelector: [
{
scheme: 'file', scheme: 'file',
language: 'python' language: 'python'
}], }
],
synchronize: { synchronize: {
// Synchronize the setting section to the server. // Synchronize the setting section to the server.
configurationSection: ['python', 'pyright'] configurationSection: ['python', 'pyright']
} }
} };
// Create the language client and start the client. // Create the language client and start the client.
const languageClient = new LanguageClient('python', 'Pyright', serverOptions, clientOptions); const languageClient = new LanguageClient('python', 'Pyright', serverOptions, clientOptions);
@ -60,18 +65,25 @@ export function activate(context: ExtensionContext) {
// Register our custom commands. // Register our custom commands.
const textEditorCommands = [Commands.orderImports, Commands.addMissingOptionalToParam]; const textEditorCommands = [Commands.orderImports, Commands.addMissingOptionalToParam];
textEditorCommands.forEach(commandName => { textEditorCommands.forEach(commandName => {
context.subscriptions.push(commands.registerTextEditorCommand(commandName, context.subscriptions.push(
commands.registerTextEditorCommand(
commandName,
(editor: TextEditor, edit: TextEditorEdit, ...args: any[]) => { (editor: TextEditor, edit: TextEditorEdit, ...args: any[]) => {
const cmd = { const cmd = {
command: commandName, command: commandName,
arguments: [editor.document.uri.toString(), ...args] arguments: [editor.document.uri.toString(), ...args]
}; };
languageClient.sendRequest('workspace/executeCommand', cmd).then((edits: TextEdit[] | undefined) => { languageClient
.sendRequest('workspace/executeCommand', cmd)
.then((edits: TextEdit[] | undefined) => {
if (edits && edits.length > 0) { if (edits && edits.length > 0) {
editor.edit(editBuilder => { editor.edit(editBuilder => {
edits.forEach(edit => { edits.forEach(edit => {
const startPos = new Position(edit.range.start.line, edit.range.start.character); const startPos = new Position(
edit.range.start.line,
edit.range.start.character
);
const endPos = new Position(edit.range.end.line, edit.range.end.character); const endPos = new Position(edit.range.end.line, edit.range.end.character);
const range = new Range(startPos, endPos); const range = new Range(startPos, endPos);
editBuilder.replace(range, edit.newText); editBuilder.replace(range, edit.newText);
@ -82,14 +94,18 @@ export function activate(context: ExtensionContext) {
}, },
() => { () => {
// Error received. For now, do nothing. // Error received. For now, do nothing.
})); }
)
);
}); });
const genericCommands = [ Commands.createTypeStub, ]; const genericCommands = [Commands.createTypeStub];
genericCommands.forEach(command => { genericCommands.forEach(command => {
context.subscriptions.push(commands.registerCommand(command, (...args: any[]) => { context.subscriptions.push(
commands.registerCommand(command, (...args: any[]) => {
languageClient.sendRequest('workspace/executeCommand', { command, arguments: args }); languageClient.sendRequest('workspace/executeCommand', { command, arguments: args });
})); })
);
}); });
} }
@ -99,4 +115,3 @@ export function deactivate() {
// anything to do here. // anything to do here.
return undefined; return undefined;
} }

9
client/src/progress.ts
View File

@ -25,13 +25,16 @@ export class ProgressReporting implements Disposable {
this._resolveProgress = resolve; this._resolveProgress = resolve;
}); });
window.withProgress({ window.withProgress(
{
location: ProgressLocation.Window, location: ProgressLocation.Window,
title: '' title: ''
}, progress => { },
progress => {
this._progress = progress; this._progress = progress;
return progressPromise; return progressPromise;
}); }
);
this._primeTimeoutTimer(); this._primeTimeoutTimer();
}); });

11
docs/build-debug.md
View File

@ -1,6 +1,7 @@
## Building Pyright ## Building Pyright
To build the project: To build the project:
1. Install [nodejs](https://nodejs.org/en/) 1. Install [nodejs](https://nodejs.org/en/)
2. Open terminal window in main directory of cloned source 2. Open terminal window in main directory of cloned source
3. Execute `npm run install:all` to install dependencies 3. Execute `npm run install:all` to install dependencies
@ -8,11 +9,11 @@ To build the project:
To build the VS Code extension package: To build the VS Code extension package:
Same as above, plus Same as above, plus
1. Execute `npm run package` 1. Execute `npm run package`
The resulting package (pyright-X.Y.Z.vsix) can be found in the client directory. The resulting package (pyright-X.Y.Z.vsix) can be found in the client directory.
To install in VS Code, go to the extensions panel and choose “Install from VSIX...” from the menu, then select the package. To install in VS Code, go to the extensions panel and choose "Install from VSIX..." from the menu, then select the package.
## Running Pyright Locally ## Running Pyright Locally
@ -20,10 +21,8 @@ Once built, you can run the command-line tool directly from the built sources by
`node ./index.js` `node ./index.js`
## Debugging Pyright ## Debugging Pyright
To debug pyright, open the root source directory within VS Code. Open the debug sub-panel and choose “Pyright CLI” from the debug target menu. Click on the green “run” icon or press F5 to build and launch the command-line version in the VS Code debugger. To debug pyright, open the root source directory within VS Code. Open the debug sub-panel and choose "Pyright CLI" from the debug target menu. Click on the green "run" icon or press F5 to build and launch the command-line version in the VS Code debugger.
To debug the VS Code extension, select “Pyright Language Client” from the debug target menu. Click on the green “run” icon or press F5 to build and launch a second copy of VS Code with the extension. Within the second VS Code instance, open a python source file so the pyright extension is loaded. Return to the first instance of VS Code and select “Pyright Language Server” from the debug target menu and click the green “run” icon. This will attach the debugger to the process that hosts the type checker. You can now set breakpoints, etc.
To debug the VS Code extension, select "Pyright Language Client" from the debug target menu. Click on the green "run" icon or press F5 to build and launch a second copy of VS Code with the extension. Within the second VS Code instance, open a python source file so the pyright extension is loaded. Return to the first instance of VS Code and select "Pyright Language Server" from the debug target menu and click the green "run" icon. This will attach the debugger to the process that hosts the type checker. You can now set breakpoints, etc.

11
docs/command-line.md
View File

@ -3,7 +3,7 @@
Pyright can be run as either a VS Code extension or as a node-based command-line tool. The command-line version allows for the following options: Pyright can be run as either a VS Code extension or as a node-based command-line tool. The command-line version allows for the following options:
| Flag | Description | | Flag | Description |
| :--------------------------------- | :--------------------------------------------------- | | :------------------------------ | :------------------------------------------------ |
| --createstub IMPORT | Create type stub file(s) for import | | --createstub IMPORT | Create type stub file(s) for import |
| --dependencies | Emit import dependency information | | --dependencies | Emit import dependency information |
| -h, --help | Show help message | | -h, --help | Show help message |
@ -17,20 +17,17 @@ Pyright can be run as either a VS Code extension or as a node-based command-line
| --version | Print pyright version | | --version | Print pyright version |
| -w, --watch | Continue to run and watch for changes (3) | | -w, --watch | Continue to run and watch for changes (3) |
(1) Pyright has built-in typeshed type stubs for Python stdlib functionality. To use a different version of typeshed type stubs, specify the directory with this option. (1) Pyright has built-in typeshed type stubs for Python stdlib functionality. To use a different version of typeshed type stubs, specify the directory with this option.
(2) This option is used in conjunction with configuration file, which can refer to different virtual environments by name. For more details, refer to the [configuration](/docs/configuration.md) documentation. This allows a common config file to be checked in to the project and shared by everyone on the development team without making assumptions about the local paths to the venv directory on each developer’s computer. (2) This option is used in conjunction with configuration file, which can refer to different virtual environments by name. For more details, refer to the [configuration](/docs/configuration.md) documentation. This allows a common config file to be checked in to the project and shared by everyone on the development team without making assumptions about the local paths to the venv directory on each developer's computer.
(3) When running in watch mode, pyright will reanalyze only those files that have been modified. These “deltas” are typically much faster than the initial analysis, which needs to analyze all files in the source tree.
(3) When running in watch mode, pyright will reanalyze only those files that have been modified. These "deltas" are typically much faster than the initial analysis, which needs to analyze all files in the source tree.
# Pyright Exit Codes # Pyright Exit Codes
| Exit Code | Meaning | | Exit Code | Meaning |
| :---------- | :--------------------------------------------------------------- | | :-------- | :------------------------------------------------------- |
| 0 | No errors reported | | 0 | No errors reported |
| 1 | One or more errors reported | | 1 | One or more errors reported |
| 2 | Fatal error occurred with no errors or warnings reported | | 2 | Fatal error occurred with no errors or warnings reported |
| 3 | Config file could not be read or parsed | | 3 | Config file could not be read or parsed |

5
docs/commands.md
View File

@ -1,8 +1,9 @@
# VS Code Commands # VS Code Commands
Pyright offers the following commands, which can be invoked from VS Code’s “Command Palette”, which can be accessed from the View menu or by pressing Cmd-Shift-P. Pyright offers the following commands, which can be invoked from VS Code's "Command Palette", which can be accessed from the View menu or by pressing Cmd-Shift-P.
## Organize Imports ## Organize Imports
This command reorders all imports found in the global (module-level) scope of the source file. As recommended in PEP8, imports are grouped into three groups, each separated by an empty line. The first group includes all built-in modules, the second group includes all third-party modules, and the third group includes all local modules. This command reorders all imports found in the global (module-level) scope of the source file. As recommended in PEP8, imports are grouped into three groups, each separated by an empty line. The first group includes all built-in modules, the second group includes all third-party modules, and the third group includes all local modules.
Within each group, imports are sorted alphabetically. And within each “from X import Y” statement, the imported symbols are sorted alphabetically. Pyright also rewraps any imports that don't fit within a single line, switching to multi-line formatting. Within each group, imports are sorted alphabetically. And within each "from X import Y" statement, the imported symbols are sorted alphabetically. Pyright also rewraps any imports that don't fit within a single line, switching to multi-line formatting.

5
docs/comments.md
View File

@ -3,6 +3,7 @@
Some behaviors of pyright can be controlled through the use of comments within the source file. Some behaviors of pyright can be controlled through the use of comments within the source file.
## Type Annotations ## Type Annotations
Versions of Python prior to 3.6 did not support type annotations for variables. Pyright honors type annotations found within a comment at the end of the same line where a variable is assigned. Versions of Python prior to 3.6 did not support type annotations for variables. Pyright honors type annotations found within a comment at the end of the same line where a variable is assigned.
``` ```
@ -12,13 +13,14 @@ self._target = 3 # type: Union[int, str]
``` ```
## File-level Type Controls ## File-level Type Controls
Strict typing controls (where all supported type-checking switches generate errors) can be enabled for a file through the use of a special comment. Typically this comment is placed at or near the top of a code file on its own line. Strict typing controls (where all supported type-checking switches generate errors) can be enabled for a file through the use of a special comment. Typically this comment is placed at or near the top of a code file on its own line.
``` ```
# pyright: strict # pyright: strict
``` ```
Individual configuration settings can also be overridden on a per-file basis and combined with “strict” typing. For example, if you want to enable all type checks except for “reportPrivateUsage”, you could add the following comment: Individual configuration settings can also be overridden on a per-file basis and combined with "strict" typing. For example, if you want to enable all type checks except for "reportPrivateUsage", you could add the following comment:
``` ```
# pyright: strict, reportPrivateUsage=false # pyright: strict, reportPrivateUsage=false
@ -29,4 +31,3 @@ Diagnostic levels are also supported.
``` ```
# pyright: reportPrivateUsage=warning, reportOptionalCall=error # pyright: reportPrivateUsage=warning, reportOptionalCall=error
``` ```

66
docs/configuration.md
View File

@ -1,24 +1,24 @@
# Pyright Configuration # Pyright Configuration
Pyright offers flexible configuration options specified in a JSON-formatted text configuration. By default, the file is called “pyrightconfig.json” and is located within the root directory of your project. Multi-root workspaces (“Add Folder to Workspace…”) are supported, and each workspace root can have its own “pyrightconfig.json” file. Pyright offers flexible configuration options specified in a JSON-formatted text configuration. By default, the file is called "pyrightconfig.json" and is located within the root directory of your project. Multi-root workspaces ("Add Folder to Workspace...") are supported, and each workspace root can have its own "pyrightconfig.json" file.
Relative paths specified within the config file are relative to the config file’s location. Paths with shell variables (including `~`) are not supported. Relative paths specified within the config file are relative to the config file's location. Paths with shell variables (including `~`) are not supported.
## Master Pyright Config Options ## Master Pyright Config Options
**include** [array of paths, optional]: Paths of directories or files that should be included. If no paths are specified, pyright defaults to the directory that contains the config file. Paths may contain wildcard characters ** (a directory or multiple levels of directories), * (a sequence of zero or more characters), or ? (a single character). If no include paths are specified, the root path for the workspace is assumed. **include** [array of paths, optional]: Paths of directories or files that should be included. If no paths are specified, pyright defaults to the directory that contains the config file. Paths may contain wildcard characters \*_ (a directory or multiple levels of directories), _ (a sequence of zero or more characters), or ? (a single character). If no include paths are specified, the root path for the workspace is assumed.
**exclude** [array of paths, optional]: Paths of directories or files that should not be included. These override the includes directories, allowing specific subdirectories to be ignored. Note that files in the exclude paths may still be included in the analysis if they are referenced (imported) by source files that are not excluded. Paths may contain wildcard characters ** (a directory or multiple levels of directories), * (a sequence of zero or more characters), or ? (a single character). If no exclude paths are specified, Pyright automatically excludes the following: `**/node_modules`, `**/__pycache__`, `.venv` and `.git`. **exclude** [array of paths, optional]: Paths of directories or files that should not be included. These override the includes directories, allowing specific subdirectories to be ignored. Note that files in the exclude paths may still be included in the analysis if they are referenced (imported) by source files that are not excluded. Paths may contain wildcard characters ** (a directory or multiple levels of directories), \* (a sequence of zero or more characters), or ? (a single character). If no exclude paths are specified, Pyright automatically excludes the following: `**/node_modules`,`\*\*/**pycache**`,`.venv`and`.git`.
**ignore** [array of paths, optional]: Paths of directories or files whose diagnostic output (errors and warnings) should be suppressed even if they are an included file or within the transitive closure of an included file. Paths may contain wildcard characters ** (a directory or multiple levels of directories), * (a sequence of zero or more characters), or ? (a single character). **ignore** [array of paths, optional]: Paths of directories or files whose diagnostic output (errors and warnings) should be suppressed even if they are an included file or within the transitive closure of an included file. Paths may contain wildcard characters `**` (a directory or multiple levels of directories), `*` (a sequence of zero or more characters), or `?` (a single character).
**strict** [array of paths, optional]: Paths of directories or files that should use “strict” analysis if they are included. This is the same as manually adding a “# pyright: strict” comment. In strict mode, all type-checking rules are enabled. Paths may contain wildcard characters ** (a directory or multiple levels of directories), * (a sequence of zero or more characters), or ? (a single character). **strict** [array of paths, optional]: Paths of directories or files that should use "strict" analysis if they are included. This is the same as manually adding a "# pyright: strict" comment. In strict mode, all type-checking rules are enabled. Paths may contain wildcard characters `**` (a directory or multiple levels of directories), `*` (a sequence of zero or more characters), or `?` (a single character).
**typeshedPath** [path, optional]: Path to a directory that contains typeshed type stub files. Pyright ships with an internal copy of some typeshed type stubs (those that cover the Python stdlib packages). If you want to use a full copy of the typeshed type stubs (including those for third-party packages), you can clone the [typeshed github repo](https://github.com/python/typeshed) to a local directory and reference the location with this path. **typeshedPath** [path, optional]: Path to a directory that contains typeshed type stub files. Pyright ships with an internal copy of some typeshed type stubs (those that cover the Python stdlib packages). If you want to use a full copy of the typeshed type stubs (including those for third-party packages), you can clone the [typeshed github repo](https://github.com/python/typeshed) to a local directory and reference the location with this path.
**typingsPath** [path, optional]: Path to a directory that contains custom type stubs. Each package's type stub file(s) are expected to be in its own subdirectory. The default value of this setting is "./typings". **typingsPath** [path, optional]: Path to a directory that contains custom type stubs. Each package's type stub file(s) are expected to be in its own subdirectory. The default value of this setting is "./typings".
**venvPath** [path, optional]: Path to a directory containing one or more subdirectories, each of which contains a virtual environment. Each execution environment (see below for details) can refer to a different virtual environment. When used in conjunction with a **venv** setting (see below), pyright will search for imports in the virtual environment’s site-packages directory rather than the paths specified in PYTHONPATH. **venvPath** [path, optional]: Path to a directory containing one or more subdirectories, each of which contains a virtual environment. Each execution environment (see below for details) can refer to a different virtual environment. When used in conjunction with a **venv** setting (see below), pyright will search for imports in the virtual environment's site-packages directory rather than the paths specified in PYTHONPATH.
**venv** [string, optional]: Used in conjunction with the venvPath, specifies the virtual environment to use. Individual execution environments may override this setting. **venv** [string, optional]: Used in conjunction with the venvPath, specifies the virtual environment to use. Individual execution environments may override this setting.
@ -28,19 +28,19 @@ Relative paths specified within the config file are relative to the config file
**executionEnvironments** [array of objects, optional]: Specifies a list of execution environments (see below). Execution environments are searched from start to finish by comparing the path of a source file with the root path specified in the execution environment. **executionEnvironments** [array of objects, optional]: Specifies a list of execution environments (see below). Execution environments are searched from start to finish by comparing the path of a source file with the root path specified in the execution environment.
## Type Check Diagnostics Settings ## Type Check Diagnostics Settings
The following settings control pyright’s diagnostic output (warnings or errors). Unless otherwise specified, each diagnostic setting can specify a boolean value (`false` indicating that no error is generated and `true` indicating that an error is generated). Alternatively, a string value of `"none"`, `"warning"`, or `"error"` can be used to specify the diagnostic level.
The following settings control pyright's diagnostic output (warnings or errors). Unless otherwise specified, each diagnostic setting can specify a boolean value (`false` indicating that no error is generated and `true` indicating that an error is generated). Alternatively, a string value of `"none"`, `"warning"`, or `"error"` can be used to specify the diagnostic level.
**strictListInference** [boolean]: When inferring the type of a list, use strict type assumptions. For example, the expression `[1, 'a', 3.4]` could be inferred to be of type `List[Any]` or `List[Union[int, str, float]]`. If this setting is true, it will use the latter (stricter) type. The default value for this setting is 'false'. **strictListInference** [boolean]: When inferring the type of a list, use strict type assumptions. For example, the expression `[1, 'a', 3.4]` could be inferred to be of type `List[Any]` or `List[Union[int, str, float]]`. If this setting is true, it will use the latter (stricter) type. The default value for this setting is 'false'.
**strictDictionaryInference** [boolean]: When inferring the type of a dictionary’s keys and values, use strict type assumptions. For example, the expression `{'a': 1, 'b': 'a'}` could be inferred to be of type `Dict[str, Any]` or `Dict[str, Union[int, str]]`. If this setting is true, it will use the latter (stricter) type. The default value for this setting is 'false'. **strictDictionaryInference** [boolean]: When inferring the type of a dictionary's keys and values, use strict type assumptions. For example, the expression `{'a': 1, 'b': 'a'}` could be inferred to be of type `Dict[str, Any]` or `Dict[str, Union[int, str]]`. If this setting is true, it will use the latter (stricter) type. The default value for this setting is 'false'.
**strictParameterNoneValue** [boolean]: PEP 484 indicates that when a function parameter is assigned a default value of None, its type should implicitly be Optional even if the explicit type is not. When enabled, this rule requires that parameter type annotations use Optional explicitly in this case. The default value for this setting is 'false'. **strictParameterNoneValue** [boolean]: PEP 484 indicates that when a function parameter is assigned a default value of None, its type should implicitly be Optional even if the explicit type is not. When enabled, this rule requires that parameter type annotations use Optional explicitly in this case. The default value for this setting is 'false'.
**enableTypeIgnoreComments** [boolean]: PEP 484 defines support for "# type: ignore" comments. This switch enables or disables support for these comments. The default value for this setting is 'true'. **enableTypeIgnoreComments** [boolean]: PEP 484 defines support for "# type: ignore" comments. This switch enables or disables support for these comments. The default value for this setting is 'true'.
**reportTypeshedErrors** [boolean or string, optional]: Generate or suppress diagnostics for typeshed type stub files. In general, these type stub files should be “clean” and generate no errors. The default value for this setting is 'none'. **reportTypeshedErrors** [boolean or string, optional]: Generate or suppress diagnostics for typeshed type stub files. In general, these type stub files should be "clean" and generate no errors. The default value for this setting is 'none'.
**reportMissingImports** [boolean or string, optional]: Generate or suppress diagnostics for imports that have no corresponding imported python file or type stub file. The default value for this setting is 'none', although pyright can do a much better job of static type checking if type stub files are provided for all imports. **reportMissingImports** [boolean or string, optional]: Generate or suppress diagnostics for imports that have no corresponding imported python file or type stub file. The default value for this setting is 'none', although pyright can do a much better job of static type checking if type stub files are provided for all imports.
@ -76,9 +76,9 @@ The following settings control pyright’s diagnostic output (warnings or errors
**reportUntypedBaseClass** [boolean or string, optional]: Generate or suppress diagnostics for base classes whose type cannot be determined statically. These obscure the class type, defeating many type analysis features. The default value for this setting is 'none'. **reportUntypedBaseClass** [boolean or string, optional]: Generate or suppress diagnostics for base classes whose type cannot be determined statically. These obscure the class type, defeating many type analysis features. The default value for this setting is 'none'.
**reportUntypedNamedTuple** [boolean or string, optional]: Generate or suppress diagnostics when “namedtuple” is used rather than “NamedTuple”. The former contains no type information, whereas the latter does. The default value for this setting is 'none'. **reportUntypedNamedTuple** [boolean or string, optional]: Generate or suppress diagnostics when "namedtuple" is used rather than "NamedTuple". The former contains no type information, whereas the latter does. The default value for this setting is 'none'.
**reportPrivateUsage** [boolean or string, optional]: Generate or suppress diagnostics for incorrect usage of private or protected variables or functions. Protected class members begin with a single underscore (“_”) and can be accessed only by subclasses. Private class members begin with a double underscore but do not end in a double underscore and can be accessed only within the declaring class. Variables and functions declared outside of a class are considered private if their names start with either a single or double underscore, and they cannot be accessed outside of the declaring module. The default value for this setting is 'none'. **reportPrivateUsage** [boolean or string, optional]: Generate or suppress diagnostics for incorrect usage of private or protected variables or functions. Protected class members begin with a single underscore ("\_") and can be accessed only by subclasses. Private class members begin with a double underscore but do not end in a double underscore and can be accessed only within the declaring class. Variables and functions declared outside of a class are considered private if their names start with either a single or double underscore, and they cannot be accessed outside of the declaring module. The default value for this setting is 'none'.
**reportConstantRedefinition** [boolean or string, optional]: Generate or suppress diagnostics for attempts to redefine variables whose names are all-caps with underscores and numerals. The default value for this setting is 'none'. **reportConstantRedefinition** [boolean or string, optional]: Generate or suppress diagnostics for attempts to redefine variables whose names are all-caps with underscores and numerals. The default value for this setting is 'none'.
@ -104,13 +104,13 @@ The following settings control pyright’s diagnostic output (warnings or errors
**reportAssertAlwaysTrue** [boolean or string, optional]: Generate or suppress diagnostics for 'assert' statement that will provably always assert. This can be indicative of a programming error. The default value for this setting is 'warning'. **reportAssertAlwaysTrue** [boolean or string, optional]: Generate or suppress diagnostics for 'assert' statement that will provably always assert. This can be indicative of a programming error. The default value for this setting is 'warning'.
**reportSelfClsParameterName** [boolean or string, optional]: Generate or suppress diagnostics for a missing or misnamed “self” parameter in instance methods and “cls” parameter in class methods. Instance methods in metaclasses (classes that derive from “type”) are allowed to use “cls” for instance methods. The default value for this setting is 'warning'. **reportSelfClsParameterName** [boolean or string, optional]: Generate or suppress diagnostics for a missing or misnamed "self" parameter in instance methods and "cls" parameter in class methods. Instance methods in metaclasses (classes that derive from "type") are allowed to use "cls" for instance methods. The default value for this setting is 'warning'.
**reportImplicitStringConcatenation** [boolean or string, optional]: Generate or suppress diagnostics for two or more string literals that follow each other, indicating an implicit concatenation. This is considered a bad practice and often masks bugs such as missing commas. The default value for this setting is 'none'. **reportImplicitStringConcatenation** [boolean or string, optional]: Generate or suppress diagnostics for two or more string literals that follow each other, indicating an implicit concatenation. This is considered a bad practice and often masks bugs such as missing commas. The default value for this setting is 'none'.
## Execution Environment Options ## Execution Environment Options
Pyright allows multiple “execution environments” to be defined for different portions of your source tree. For example, a subtree may be designed to run with different import search paths or a different version of the python interpreter than the rest of the source base.
Pyright allows multiple "execution environments" to be defined for different portions of your source tree. For example, a subtree may be designed to run with different import search paths or a different version of the python interpreter than the rest of the source base.
The following settings can be specified for each execution environment. The following settings can be specified for each execution environment.
@ -124,34 +124,25 @@ The following settings can be specified for each execution environment.
**pythonPlatform** [string, optional]: Specifies the target platform that will be used for this execution environment. If not specified, the global `pythonPlatform` setting is used instead. **pythonPlatform** [string, optional]: Specifies the target platform that will be used for this execution environment. If not specified, the global `pythonPlatform` setting is used instead.
# VS Code Extension Settings # VS Code Extension Settings
Pyright will import the following settings set through VS Code. These override the values provided in the configuration file. Pyright will import the following settings set through VS Code. These override the values provided in the configuration file.
**python.venvPath**: Same as the **venvPath** setting described above. **python.venvPath**: Same as the **venvPath** setting described above.
**python.analysis.typeshedPaths**: An array of typeshed paths to search. Pyright supports only one such path. If provided in the VS Code setting, the first entry overrides the **typeshedPath** configuration file entry described above. **python.analysis.typeshedPaths**: An array of typeshed paths to search. Pyright supports only one such path. If provided in the VS Code setting, the first entry overrides the **typeshedPath** configuration file entry described above.
## Sample Config File ## Sample Config File
The following is an example of a pyright config file: The following is an example of a pyright config file:
```json ```json
{ {
"include": [ "include": ["src"],
"src"
],
"exclude": [ "exclude": ["**/node_modules", "**/__pycache__", "src/experimental", "src/web/node_modules", "src/typestubs"],
"**/node_modules",
"**/__pycache__",
"src/experimental",
"src/web/node_modules",
"src/typestubs"
],
"ignore": [ "ignore": ["src/oldstuff"],
"src/oldstuff"
],
"typingsPath": "src/typestubs", "typingsPath": "src/typestubs",
"venvPath": "/home/foo/.venvs", "venvPath": "/home/foo/.venvs",
@ -168,24 +159,17 @@ The following is an example of a pyright config file:
"root": "src/web", "root": "src/web",
"pythonVersion": "3.5", "pythonVersion": "3.5",
"pythonPlatform": "Windows", "pythonPlatform": "Windows",
"extraPaths": [ "extraPaths": ["src/service_libs"]
"src/service_libs"
]
}, },
{ {
"root": "src/sdk", "root": "src/sdk",
"pythonVersion": "3.0", "pythonVersion": "3.0",
"extraPaths": [ "extraPaths": ["src/backend"],
"src/backend"
],
"venv": "venv_bar" "venv": "venv_bar"
}, },
{ {
"root": "src/tests", "root": "src/tests",
"extraPaths": [ "extraPaths": ["src/tests/e2e", "src/sdk"]
"src/tests/e2e",
"src/sdk"
]
}, },
{ {
"root": "src" "root": "src"

8
docs/getting-started.md
View File

@ -3,12 +3,12 @@
A static type checker like pyright can add incremental value to your source code as more type information is provided. A static type checker like pyright can add incremental value to your source code as more type information is provided.
Here is a typical progression: Here is a typical progression:
1. Install pyright (either the VS Code extension or command-line tool). 1. Install pyright (either the VS Code extension or command-line tool).
2. Write a minimal `pyrightconfig.json` that defines `include` entries. Place the config file in your project’s top-level directory and commit it to your repo. 2. Write a minimal `pyrightconfig.json` that defines `include` entries. Place the config file in your project's top-level directory and commit it to your repo.
3. Run pyright over your source base with the default settings. Fix any errors and warnings that it emits. 3. Run pyright over your source base with the default settings. Fix any errors and warnings that it emits.
4. Enable the `reportMissingTypeStubs` setting in the config file and add (minimal) type stub files for the imported files. You may wish to create a “typestubs” directory within your code base -- a common location for all of your custom type stub files. You may be able to find preexisting type stub files for some of your imports within the typeshed repo (in the [third-party directory](https://github.com/python/typeshed/tree/master/third_party)). 4. Enable the `reportMissingTypeStubs` setting in the config file and add (minimal) type stub files for the imported files. You may wish to create a "typestubs" directory within your code base -- a common location for all of your custom type stub files. You may be able to find preexisting type stub files for some of your imports within the typeshed repo (in the [third-party directory](https://github.com/python/typeshed/tree/master/third_party)).
5. Check in your custom type stub files and configure pyright to run as part of your continuous integration (CI) environment to keep the project “type clean”. 5. Check in your custom type stub files and configure pyright to run as part of your continuous integration (CI) environment to keep the project "type clean".
6. Incrementally add type annotations to your code files. The annotations that provide most value are on function input parameters, instance variables, and return parameters (in that order). Note that annotation of variables (instance, class and local) requires Python 3.6 or newer. 6. Incrementally add type annotations to your code files. The annotations that provide most value are on function input parameters, instance variables, and return parameters (in that order). Note that annotation of variables (instance, class and local) requires Python 3.6 or newer.
7. Enable stricter type checking options like "reportOptionalSubscript", "reportOptionalMemberAccess", "reportOptionalCall", and "reportUntypedFunctionDecorator". 7. Enable stricter type checking options like "reportOptionalSubscript", "reportOptionalMemberAccess", "reportOptionalCall", and "reportUntypedFunctionDecorator".
8. On a file-by-file basis, enable all type checking options by adding the comment `# pyright: strict` somewhere in the file. 8. On a file-by-file basis, enable all type checking options by adding the comment `# pyright: strict` somewhere in the file.

7
docs/import-resolution.md
View File

@ -2,9 +2,8 @@
Pyright resolves external imports based on several configuration settings. If a venvPath and venv are specified, these are used to locate the `site-packages` directory within the virtual environment. Pyright resolves external imports based on several configuration settings. If a venvPath and venv are specified, these are used to locate the `site-packages` directory within the virtual environment.
If no venvPath is specified, Pyright falls back to the paths found in the default python interpreter’s search paths (or the python interpreter pointed to by the “python.pythonPath” setting in VS Code). Only directory-based paths are supported (as opposed to zip files or other loader packages). If no venvPath is specified, Pyright falls back to the paths found in the default python interpreter's search paths (or the python interpreter pointed to by the "python.pythonPath" setting in VS Code). Only directory-based paths are supported (as opposed to zip files or other loader packages).
The Pyright configuration file supports “execution environment” definitions, each of which can define additional paths. These are searched in addition to the venv or PYTHONPATH directories. The Pyright configuration file supports "execution environment" definitions, each of which can define additional paths. These are searched in addition to the venv or PYTHONPATH directories.
If Pyright is reporting import resolution errors, additional diagnostic information may help you determine why. If you are using the command-line version, try adding the “--verbose” switch. If you are using the VS Code extension, look at the “Output” window (View -> Output) and choose the “Pyright” view from the popup menu.
If Pyright is reporting import resolution errors, additional diagnostic information may help you determine why. If you are using the command-line version, try adding the "--verbose" switch. If you are using the VS Code extension, look at the "Output" window (View -> Output) and choose the "Pyright" view from the popup menu.

33
docs/internals.md
View File

@ -2,15 +2,14 @@
## Code Structure ## Code Structure
* client/src/extension.ts: Language Server Protocol (LSP) client entry point for VS Code extension. - client/src/extension.ts: Language Server Protocol (LSP) client entry point for VS Code extension.
* client/typeshed-fallback/: Recent copy of Typeshed type stub files for Python stdlib - client/typeshed-fallback/: Recent copy of Typeshed type stub files for Python stdlib
* server/src/pyright.ts: Main entry point for command-line tool - server/src/pyright.ts: Main entry point for command-line tool
* server/src/server.ts: Main entry point for LSP server - server/src/server.ts: Main entry point for LSP server
* server/src/analyzer: Modules that perform analysis passes over Python parse tree - server/src/analyzer: Modules that perform analysis passes over Python parse tree
* server/src/common: Modules that are common to the parser and analyzer - server/src/common: Modules that are common to the parser and analyzer
* server/src/parser: Modules that perform tokenization and parsing of Python source - server/src/parser: Modules that perform tokenization and parsing of Python source
* server/src/tests: Tests for the parser and analyzer - server/src/tests: Tests for the parser and analyzer
## Core Concepts ## Core Concepts
@ -22,12 +21,11 @@ The program tracks multiple [sourceFile](https://github.com/Microsoft/pyright/bl
The program makes use of an [importResolver](https://github.com/Microsoft/pyright/blob/master/server/src/analyzer/importResolver.ts) to resolve the imported modules referenced within each source file. The program makes use of an [importResolver](https://github.com/Microsoft/pyright/blob/master/server/src/analyzer/importResolver.ts) to resolve the imported modules referenced within each source file.
## Analysis Phases ## Analysis Phases
Pyright performs the following analysis phases for each source file. Pyright performs the following analysis phases for each source file.
The [tokenizer](https://github.com/Microsoft/pyright/blob/master/server/src/parser/tokenizer.ts) is responsible for converting the file’s string contents into a stream of tokens. White space, comments, and some end-of-line characters are ignored, as they are not needed by the parser. The [tokenizer](https://github.com/Microsoft/pyright/blob/master/server/src/parser/tokenizer.ts) is responsible for converting the file's string contents into a stream of tokens. White space, comments, and some end-of-line characters are ignored, as they are not needed by the parser.
The [parser](https://github.com/Microsoft/pyright/blob/master/server/src/parser/parser.ts) is responsible for converting the token stream into a parse tree. A generalized [parseTreeWalker](https://github.com/Microsoft/pyright/blob/master/server/src/analyzer/parseTreeWalker.ts) provides a convenient way to traverse the parse tree. All subsequent analysis phases utilize the parseTreeWalker. The [parser](https://github.com/Microsoft/pyright/blob/master/server/src/parser/parser.ts) is responsible for converting the token stream into a parse tree. A generalized [parseTreeWalker](https://github.com/Microsoft/pyright/blob/master/server/src/analyzer/parseTreeWalker.ts) provides a convenient way to traverse the parse tree. All subsequent analysis phases utilize the parseTreeWalker.
@ -37,11 +35,12 @@ The [checker](https://github.com/Microsoft/pyright/blob/master/server/src/analyz
## Type Checking Concepts ## Type Checking Concepts
Pyright uses an internal type called “Unknown” to represent types that are not annotated and cannot be inferred. Unknown is generally treated like the “Any” type in terms of type checking, but it provides a way for developers to know when type annotations are missing and could provide additional value. Pyright uses an internal type called "Unknown" to represent types that are not annotated and cannot be inferred. Unknown is generally treated like the "Any" type in terms of type checking, but it provides a way for developers to know when type annotations are missing and could provide additional value.
Pyright attempts to infer the types of global (module-level) variables, class variables, instance variables, and local variables. Return and yield types are also inferred. If type annotations are provided in these cases, the type annotation overrides any inferred types. Pyright attempts to infer the types of global (module-level) variables, class variables, instance variables, and local variables. Return and yield types are also inferred. If type annotations are provided in these cases, the type annotation overrides any inferred types.
Pyright supports type constraints (sometimes called “path constraints” or "type guards") to track assumptions that apply within certain code flow paths. For example, consider the following code: Pyright supports type constraints (sometimes called "path constraints" or "type guards") to track assumptions that apply within certain code flow paths. For example, consider the following code:
```python ```python
def (a: Optional[Union[str, List[str]]): def (a: Optional[Union[str, List[str]]):
if isinstance(a, str): if isinstance(a, str):
@ -55,6 +54,7 @@ def (a: Optional[Union[str, List[str]]):
In this example, the type evaluator knows that parameter a is either None, str, or List[str]. Within the first `if` clause, a is constrained to be a str. Within the `elif` clause, it is constrained to be a List[str], and within the `else` clause, it has to be None (by process of elimination). The type checker would therefore flag the final line as an error if the log method could not accept None as a parameter. In this example, the type evaluator knows that parameter a is either None, str, or List[str]. Within the first `if` clause, a is constrained to be a str. Within the `elif` clause, it is constrained to be a List[str], and within the `else` clause, it has to be None (by process of elimination). The type checker would therefore flag the final line as an error if the log method could not accept None as a parameter.
If the type constraint logic exhausts all possible subtypes, it can be assumed that a code path will never be taken. For example, consider the following: If the type constraint logic exhausts all possible subtypes, it can be assumed that a code path will never be taken. For example, consider the following:
```python ```python
def (a: Union[Foo, Bar]): def (a: Union[Foo, Bar]):
if isinstance(a, Foo): if isinstance(a, Foo):
@ -68,15 +68,16 @@ def (a: Union[Foo, Bar]):
a.do_something_3() a.do_something_3()
``` ```
In this case, the type of parameter “a” is initially “Union[Foo, Bar]”. Within the “if” clause, the type constraint logic will conclude that it must be of type “Foo”. Within the “elif” clause, it must be of type “Bar”. What type is it within the “else” clause? The type constraint system has eliminated all possible subtypes, so it gives it the type “Never”. This is generally indicates that there’s a logic error in the code because there’s way that code block will ever be executed. In this case, the type of parameter "a" is initially "Union[Foo, Bar]". Within the "if" clause, the type constraint logic will conclude that it must be of type "Foo". Within the "elif" clause, it must be of type "Bar". What type is it within the "else" clause? The type constraint system has eliminated all possible subtypes, so it gives it the type "Never". This is generally indicates that there's a logic error in the code because there's way that code block will ever be executed.
## Type Inference ## Type Inference
In cases where explicit type annotations are not provided, Pyright attempts to infer the types. The inferred return type of a function is determined from all of the return (and yield) statements within the function’s definition. The inferred type of a local variable is determined by the expression that is assigned to that variable. Likewise, the type of a member variable is inferred from all assignments to that member variable within its defining class. In cases where explicit type annotations are not provided, Pyright attempts to infer the types. The inferred return type of a function is determined from all of the return (and yield) statements within the function's definition. The inferred type of a local variable is determined by the expression that is assigned to that variable. Likewise, the type of a member variable is inferred from all assignments to that member variable within its defining class.
The types of input parameters cannot be inferred, with the exception of the “self” or “cls” parameter for instance members and class members, respectively. The types of input parameters cannot be inferred, with the exception of the "self" or "cls" parameter for instance members and class members, respectively.
If an inferred return type is unknown or partially unknown because input parameter types are not annotated, Pyright may still be able to infer the return type based on the types of arguments at the call site. If an inferred return type is unknown or partially unknown because input parameter types are not annotated, Pyright may still be able to infer the return type based on the types of arguments at the call site.
``` ```
def add_values(a, b): def add_values(a, b):
return a + b return a + b

4
docs/settings.md
View File

@ -2,7 +2,7 @@
The Pyright VS Code extension honors the following settings. The Pyright VS Code extension honors the following settings.
**pyright.disableLanguageServices** [boolean]: Disables all language services except for “hover”. This includes type completion, signature completion, find definition, find references, and find symbols in file. This option is useful if you want to use pyright only as a type checker but want to run another Python language server for langue service features. **pyright.disableLanguageServices** [boolean]: Disables all language services except for "hover". This includes type completion, signature completion, find definition, find references, and find symbols in file. This option is useful if you want to use pyright only as a type checker but want to run another Python language server for langue service features.
**pyright.openFilesOnly** [boolean]: Determines whether pyright analyzes (and reports errors for) all files in the workspace, as indicated by the config file. If this option is set to true, pyright analyzes only open files. **pyright.openFilesOnly** [boolean]: Determines whether pyright analyzes (and reports errors for) all files in the workspace, as indicated by the config file. If this option is set to true, pyright analyzes only open files.
@ -13,5 +13,3 @@ The Pyright VS Code extension honors the following settings.
**python.pythonPath** [path]: Path to Python interpreter. **python.pythonPath** [path]: Path to Python interpreter.
**python.venvPath** [path]: Path to folder with subdirectories that contain virtual environments. **python.venvPath** [path]: Path to folder with subdirectories that contain virtual environments.

27
docs/type-stubs.md
View File

@ -1,28 +1,30 @@
# Type Stub Files # Type Stub Files
Type stubs are “.pyi” files that specify the public interface for a library. They use a variant of the Python syntax that allows for “...” to be used in place of any implementation details. Type stubs define the public contract for the library. Type stubs are ".pyi" files that specify the public interface for a library. They use a variant of the Python syntax that allows for "..." to be used in place of any implementation details. Type stubs define the public contract for the library.
## Importance of Type Stub Files ## Importance of Type Stub Files
Regardless of the search path, Pyright always attempts to resolve an import with a type stub (“.pyi”) file before falling back to a python source (“.py”) file. If a type stub cannot be located for an external import, that import will be treated as a “black box” for purposes of type analysis. Any symbols imported from these modules will be of type “Unknown”, and wildcard imports (of the form `from foo import *`) will not populate the module’s namespace with specific symbol names. Regardless of the search path, Pyright always attempts to resolve an import with a type stub (".pyi") file before falling back to a python source (".py") file. If a type stub cannot be located for an external import, that import will be treated as a "black box" for purposes of type analysis. Any symbols imported from these modules will be of type "Unknown", and wildcard imports (of the form `from foo import *`) will not populate the module's namespace with specific symbol names.
Why does Pyright not attempt (by default) to determine types from imported python sources? There are several reasons. Why does Pyright not attempt (by default) to determine types from imported python sources? There are several reasons.
1. Imported libraries can be quite large, so analyzing them can require significant time and computation. 1. Imported libraries can be quite large, so analyzing them can require significant time and computation.
2. Some libraries are thin shims on top of native (C++) libraries. Little or no type information would be inferable in these cases. 2. Some libraries are thin shims on top of native (C++) libraries. Little or no type information would be inferable in these cases.
3. Some libraries override Python’s default loader logic. Static analysis is not possible in these cases. 3. Some libraries override Python's default loader logic. Static analysis is not possible in these cases.
4. Type information inferred from source files is often of low value because many types cannot be inferred correctly. Even if concrete types can be inferred, generic type definitions cannot. 4. Type information inferred from source files is often of low value because many types cannot be inferred correctly. Even if concrete types can be inferred, generic type definitions cannot.
5. Type analysis would expose all symbols from an imported module, even those that are not meant to be exposed by the author. Unlike many other languages, Python offers no way of differentiating between a symbol that is meant to be exported and one that isn’t. 5. Type analysis would expose all symbols from an imported module, even those that are not meant to be exposed by the author. Unlike many other languages, Python offers no way of differentiating between a symbol that is meant to be exported and one that isn't.
If you’re serious about static type checking for your Python source base, it’s highly recommended that you use type stub files for all external imports. If you are unable to find a type stub for a particular library, the recommended approach is to create a custom type stub file that defines the portion of that module’s interface used by your code. More library authors have started to provide type stub files. If you're serious about static type checking for your Python source base, it's highly recommended that you use type stub files for all external imports. If you are unable to find a type stub for a particular library, the recommended approach is to create a custom type stub file that defines the portion of that module's interface used by your code. More library authors have started to provide type stub files.
## Generating Type Stubs ## Generating Type Stubs
If you use only a few classes, methods or functions within a library, writing a type stub file by hand is feasible. For large libraries, this can become tedious and error-prone. Pyright can generate “draft” versions of type stub files for you.
To generate a type stub file from within VS Code, enable the reportMissingTypeStubs” setting in your pyrightconfig.json file or by adding a comment `# pyright: reportMissingTypeStubs=true` to individual source files. Make sure you have the target library installed in the python environment that pyright is configured to use for import resolution. Optionally specify a “typingsPath” in your pyrightconfig.json file. This is where pyright will generate your type stub files. By default, the typingsPath is set to "./typings". If you use only a few classes, methods or functions within a library, writing a type stub file by hand is feasible. For large libraries, this can become tedious and error-prone. Pyright can generate "draft" versions of type stub files for you.
To generate a type stub file from within VS Code, enable the reportMissingTypeStubs" setting in your pyrightconfig.json file or by adding a comment `# pyright: reportMissingTypeStubs=true` to individual source files. Make sure you have the target library installed in the python environment that pyright is configured to use for import resolution. Optionally specify a "typingsPath" in your pyrightconfig.json file. This is where pyright will generate your type stub files. By default, the typingsPath is set to "./typings".
### Generating Type Stubs in VS Code ### Generating Type Stubs in VS Code
If “reportMissingTypeStubs” is enabled, pyright will highlight any imports that have no type stub. Hover over the error message, and you will see a “Quick Fix” link. Clicking on this link will reveal a popup menu item titled “Create Type Stub For XXX”. The example below shows a missing typestub for the `django` library.
If "reportMissingTypeStubs" is enabled, pyright will highlight any imports that have no type stub. Hover over the error message, and you will see a "Quick Fix" link. Clicking on this link will reveal a popup menu item titled "Create Type Stub For XXX". The example below shows a missing typestub for the `django` library.
![Pyright](/docs/img/CreateTypeStub1.png) ![Pyright](/docs/img/CreateTypeStub1.png)
@ -31,21 +33,23 @@ Click on the menu item to create the type stub. Depending on the size of the lib
![Pyright](/docs/img/CreateTypeStub2.png) ![Pyright](/docs/img/CreateTypeStub2.png)
### Generating Type Stubs from Command Line ### Generating Type Stubs from Command Line
The command-line version of pyright can also be used to generate type stubs. As with the VS Code version, it must be run within the context of your configured project. Then type `pyright --createstub [import-name]`. The command-line version of pyright can also be used to generate type stubs. As with the VS Code version, it must be run within the context of your configured project. Then type `pyright --createstub [import-name]`.
For example: For example:
`pyright --createstub django` `pyright --createstub django`
### Cleaning Up Generated Type Stubs ### Cleaning Up Generated Type Stubs
Pyright can give you a head start by creating type stubs, but you will typically need to clean up the first draft, fixing various errors and omissions that pyright was not able to infer from the original library code. Pyright can give you a head start by creating type stubs, but you will typically need to clean up the first draft, fixing various errors and omissions that pyright was not able to infer from the original library code.
A few common situations that need to be cleaned up: A few common situations that need to be cleaned up:
1. When generating a “.pyi” file, pyright removes any imports that are not referenced. Sometimes libraries import symbols that are meant to be simply re-exported from a module even though they are not referenced internally to that module. In such cases, you will need to manually add back these imports. Pyright does not perform this import culling in `__init__.pyi` files because this re-export technique is especially common in such files. 1. When generating a ".pyi" file, pyright removes any imports that are not referenced. Sometimes libraries import symbols that are meant to be simply re-exported from a module even though they are not referenced internally to that module. In such cases, you will need to manually add back these imports. Pyright does not perform this import culling in `__init__.pyi` files because this re-export technique is especially common in such files.
2. Some libraries attempt to import modules within a try statement. These constructs don’t work well in type stub files because they cannot be evaluated statically. Pyright omits any try statements when creating “.pyi” files, so you may need to add back in these import statements. 2. Some libraries attempt to import modules within a try statement. These constructs don't work well in type stub files because they cannot be evaluated statically. Pyright omits any try statements when creating ".pyi" files, so you may need to add back in these import statements.
3. Decorator functions are especially problematic for static type analyzers. Unless properly typed, they completely hide the signature of any class or function they are applied to. For this reason, it is highly recommended that you enable the “reportUntypedFunctionDecorator” and “reportUntypedClassDecorator” switches in pyrightconfig.json. Most decorators simply return the same function they are passed. Those can easily be annotated with a TypeVar like this: 3. Decorator functions are especially problematic for static type analyzers. Unless properly typed, they completely hide the signature of any class or function they are applied to. For this reason, it is highly recommended that you enable the "reportUntypedFunctionDecorator" and "reportUntypedClassDecorator" switches in pyrightconfig.json. Most decorators simply return the same function they are passed. Those can easily be annotated with a TypeVar like this:
``` ```
from typings import Any, Callable, TypeVar from typings import Any, Callable, TypeVar
@ -54,4 +58,3 @@ _FuncT = TypeVar('_FuncT', bound=Callable[..., Any])
def my_decorator(*args, **kw) -> Callable[[_FuncT], _FuncT]: ... def my_decorator(*args, **kw) -> Callable[[_FuncT], _FuncT]: ...
``` ```

2
index.debug.js
View File

@ -3,4 +3,4 @@
// Stash the base directory into a global variable. // Stash the base directory into a global variable.
global.__rootDirectory = __dirname + '/dist/'; global.__rootDirectory = __dirname + '/dist/';
require('./client/server/pyright') require('./client/server/pyright');

2
index.js
View File

@ -3,4 +3,4 @@
// Stash the base directory into a global variable. // Stash the base directory into a global variable.
global.__rootDirectory = __dirname + '/dist/'; global.__rootDirectory = __dirname + '/dist/';
require('./dist/pyright') require('./dist/pyright');

85
package-lock.json generated
View File

@ -74,9 +74,9 @@
} }
}, },
"@types/json-schema": { "@types/json-schema": {
"version": "7.0.3", "version": "7.0.4",
"resolved": "https://registry.npmjs.org/@types/json-schema/-/json-schema-7.0.3.tgz", "resolved": "https://registry.npmjs.org/@types/json-schema/-/json-schema-7.0.4.tgz",
"integrity": "sha512-Il2DtDVRGDcqjDtE+rF8iqg1CArehSK84HZJCT7AMITlyXRBpuPhqGLDQMowraqqu1coEaimg4ZOqggt6L6L+A==", "integrity": "sha512-8+KAKzEvSUdeo+kmqnKrqgeE+LcA0tjYWFY7RPProVYwnqDjukzO+3b6dLD56rYX5TdWejnEOLJYOIeh4CXKuA==",
"dev": true "dev": true
}, },
"@types/minimatch": { "@types/minimatch": {
@ -85,25 +85,19 @@
"integrity": "sha512-tHq6qdbT9U1IRSGf14CL0pUlULksvY9OZ+5eEgl1N7t+OA3tGvNpxJCzuKQlsNgCVwbAs670L1vcVQi8j9HjnA==", "integrity": "sha512-tHq6qdbT9U1IRSGf14CL0pUlULksvY9OZ+5eEgl1N7t+OA3tGvNpxJCzuKQlsNgCVwbAs670L1vcVQi8j9HjnA==",
"dev": true "dev": true
}, },
"@types/mocha": {
"version": "5.2.7",
"resolved": "https://registry.npmjs.org/@types/mocha/-/mocha-5.2.7.tgz",
"integrity": "sha512-NYrtPht0wGzhwe9+/idPaBB+TqkY9AhTvOLMkThm0IoEfLaiVQZwBwyJ5puCkO3AUCWrmcoePjp2mbFocKy4SQ==",
"dev": true
},
"@types/node": { "@types/node": {
"version": "12.12.21", "version": "12.12.28",
"resolved": "https://registry.npmjs.org/@types/node/-/node-12.12.21.tgz", "resolved": "https://registry.npmjs.org/@types/node/-/node-12.12.28.tgz",
"integrity": "sha512-8sRGhbpU+ck1n0PGAUgVrWrWdjSW2aqNeyC15W88GRsMpSwzv6RJGlLhE7s2RhVSOdyDmxbqlWSeThq4/7xqlA==", "integrity": "sha512-g73GJYJDXgf0jqg+P9S8h2acWbDXNkoCX8DLtJVu7Fkn788pzQ/oJsrdJz/2JejRf/SjfZaAhsw+3nd1D5EWGg==",
"dev": true "dev": true
}, },
"@typescript-eslint/eslint-plugin": { "@typescript-eslint/eslint-plugin": {
"version": "2.12.0", "version": "2.21.0",
"resolved": "https://registry.npmjs.org/@typescript-eslint/eslint-plugin/-/eslint-plugin-2.12.0.tgz", "resolved": "https://registry.npmjs.org/@typescript-eslint/eslint-plugin/-/eslint-plugin-2.21.0.tgz",
"integrity": "sha512-1t4r9rpLuEwl3hgt90jY18wJHSyb0E3orVL3DaqwmpiSDHmHiSspVsvsFF78BJ/3NNG3qmeso836jpuBWYziAA==", "integrity": "sha512-b5jjjDMxzcjh/Sbjuo7WyhrQmVJg0WipTHQgXh5Xwx10uYm6nPWqN1WGOsaNq4HR3Zh4wUx4IRQdDkCHwyewyw==",
"dev": true, "dev": true,
"requires": { "requires": {
"@typescript-eslint/experimental-utils": "2.12.0", "@typescript-eslint/experimental-utils": "2.21.0",
"eslint-utils": "^1.4.3", "eslint-utils": "^1.4.3",
"functional-red-black-tree": "^1.0.1", "functional-red-black-tree": "^1.0.1",
"regexpp": "^3.0.0", "regexpp": "^3.0.0",
@ -111,39 +105,39 @@
} }
}, },
"@typescript-eslint/experimental-utils": { "@typescript-eslint/experimental-utils": {
"version": "2.12.0", "version": "2.21.0",
"resolved": "https://registry.npmjs.org/@typescript-eslint/experimental-utils/-/experimental-utils-2.12.0.tgz", "resolved": "https://registry.npmjs.org/@typescript-eslint/experimental-utils/-/experimental-utils-2.21.0.tgz",
"integrity": "sha512-jv4gYpw5N5BrWF3ntROvCuLe1IjRenLy5+U57J24NbPGwZFAjhnM45qpq0nDH1y/AZMb3Br25YiNVwyPbz6RkA==", "integrity": "sha512-olKw9JP/XUkav4lq0I7S1mhGgONJF9rHNhKFn9wJlpfRVjNo3PPjSvybxEldvCXnvD+WAshSzqH5cEjPp9CsBA==",
"dev": true, "dev": true,
"requires": { "requires": {
"@types/json-schema": "^7.0.3", "@types/json-schema": "^7.0.3",
"@typescript-eslint/typescript-estree": "2.12.0", "@typescript-eslint/typescript-estree": "2.21.0",
"eslint-scope": "^5.0.0" "eslint-scope": "^5.0.0"
} }
}, },
"@typescript-eslint/parser": { "@typescript-eslint/parser": {
"version": "2.12.0", "version": "2.21.0",
"resolved": "https://registry.npmjs.org/@typescript-eslint/parser/-/parser-2.12.0.tgz", "resolved": "https://registry.npmjs.org/@typescript-eslint/parser/-/parser-2.21.0.tgz",
"integrity": "sha512-lPdkwpdzxEfjI8TyTzZqPatkrswLSVu4bqUgnB03fHSOwpC7KSerPgJRgIAf11UGNf7HKjJV6oaPZI4AghLU6g==", "integrity": "sha512-VrmbdrrrvvI6cPPOG7uOgGUFXNYTiSbnRq8ZMyuGa4+qmXJXVLEEz78hKuqupvkpwJQNk1Ucz1TenrRP90gmBg==",
"dev": true, "dev": true,
"requires": { "requires": {
"@types/eslint-visitor-keys": "^1.0.0", "@types/eslint-visitor-keys": "^1.0.0",
"@typescript-eslint/experimental-utils": "2.12.0", "@typescript-eslint/experimental-utils": "2.21.0",
"@typescript-eslint/typescript-estree": "2.12.0", "@typescript-eslint/typescript-estree": "2.21.0",
"eslint-visitor-keys": "^1.1.0" "eslint-visitor-keys": "^1.1.0"
} }
}, },
"@typescript-eslint/typescript-estree": { "@typescript-eslint/typescript-estree": {
"version": "2.12.0", "version": "2.21.0",
"resolved": "https://registry.npmjs.org/@typescript-eslint/typescript-estree/-/typescript-estree-2.12.0.tgz", "resolved": "https://registry.npmjs.org/@typescript-eslint/typescript-estree/-/typescript-estree-2.21.0.tgz",
"integrity": "sha512-rGehVfjHEn8Frh9UW02ZZIfJs6SIIxIu/K1bbci8rFfDE/1lQ8krIJy5OXOV3DVnNdDPtoiPOdEANkLMrwXbiQ==", "integrity": "sha512-NC/nogZNb9IK2MEFQqyDBAciOT8Lp8O3KgAfvHx2Skx6WBo+KmDqlU3R9KxHONaijfTIKtojRe3SZQyMjr3wBw==",
"dev": true, "dev": true,
"requires": { "requires": {
"debug": "^4.1.1", "debug": "^4.1.1",
"eslint-visitor-keys": "^1.1.0", "eslint-visitor-keys": "^1.1.0",
"glob": "^7.1.6", "glob": "^7.1.6",
"is-glob": "^4.0.1", "is-glob": "^4.0.1",
"lodash.unescape": "4.0.1", "lodash": "^4.17.15",
"semver": "^6.3.0", "semver": "^6.3.0",
"tsutils": "^3.17.1" "tsutils": "^3.17.1"
} }
@ -544,6 +538,15 @@
} }
} }
}, },
"eslint-config-prettier": {
"version": "6.10.0",
"resolved": "https://registry.npmjs.org/eslint-config-prettier/-/eslint-config-prettier-6.10.0.tgz",
"integrity": "sha512-AtndijGte1rPILInUdHjvKEGbIV06NuvPrqlIEaEaWtbtvJh464mDeyGMdZEQMsGvC0ZVkiex1fSNcC4HAbRGg==",
"dev": true,
"requires": {
"get-stdin": "^6.0.0"
}
},
"eslint-plugin-simple-import-sort": { "eslint-plugin-simple-import-sort": {
"version": "5.0.1", "version": "5.0.1",
"resolved": "https://registry.npmjs.org/eslint-plugin-simple-import-sort/-/eslint-plugin-simple-import-sort-5.0.1.tgz", "resolved": "https://registry.npmjs.org/eslint-plugin-simple-import-sort/-/eslint-plugin-simple-import-sort-5.0.1.tgz",
@ -738,6 +741,12 @@
"integrity": "sha1-GwqzvVU7Kg1jmdKcDj6gslIHgyc=", "integrity": "sha1-GwqzvVU7Kg1jmdKcDj6gslIHgyc=",
"dev": true "dev": true
}, },
"get-stdin": {
"version": "6.0.0",
"resolved": "https://registry.npmjs.org/get-stdin/-/get-stdin-6.0.0.tgz",
"integrity": "sha512-jp4tHawyV7+fkkSKyvjuLZswblUtz+SQKzSWnBbii16BuZksJlU1wuBYXY75r+duh/llF1ur6oNwi+2ZzjKZ7g==",
"dev": true
},
"glob": { "glob": {
"version": "7.1.6", "version": "7.1.6",
"resolved": "https://registry.npmjs.org/glob/-/glob-7.1.6.tgz", "resolved": "https://registry.npmjs.org/glob/-/glob-7.1.6.tgz",
@ -1021,12 +1030,6 @@
"integrity": "sha512-8xOcRHvCjnocdS5cpwXQXVzmmh5e5+saE2QGoeQmbKmRS6J3VQppPOIt0MnmE+4xlZoumy0GPG0D0MVIQbNA1A==", "integrity": "sha512-8xOcRHvCjnocdS5cpwXQXVzmmh5e5+saE2QGoeQmbKmRS6J3VQppPOIt0MnmE+4xlZoumy0GPG0D0MVIQbNA1A==",
"dev": true "dev": true
}, },
"lodash.unescape": {
"version": "4.0.1",
"resolved": "https://registry.npmjs.org/lodash.unescape/-/lodash.unescape-4.0.1.tgz",
"integrity": "sha1-vyJJiGzlFM2hEvrpIYzcBlIR/Jw=",
"dev": true
},
"loud-rejection": { "loud-rejection": {
"version": "1.6.0", "version": "1.6.0",
"resolved": "https://registry.npmjs.org/loud-rejection/-/loud-rejection-1.6.0.tgz", "resolved": "https://registry.npmjs.org/loud-rejection/-/loud-rejection-1.6.0.tgz",
@ -1298,6 +1301,12 @@
"integrity": "sha1-IZMqVJ9eUv/ZqCf1cOBL5iqX2lQ=", "integrity": "sha1-IZMqVJ9eUv/ZqCf1cOBL5iqX2lQ=",
"dev": true "dev": true
}, },
"prettier": {
"version": "1.19.1",
"resolved": "https://registry.npmjs.org/prettier/-/prettier-1.19.1.tgz",
"integrity": "sha512-s7PoyDv/II1ObgQunCbB9PdLmUcBZcnWOcxDh7O0N/UwDEsHyqkW+Qh28jW+mVuCdx7gLB0BotYI1Y6uI9iyew==",
"dev": true
},
"progress": { "progress": {
"version": "2.0.3", "version": "2.0.3",
"resolved": "https://registry.npmjs.org/progress/-/progress-2.0.3.tgz", "resolved": "https://registry.npmjs.org/progress/-/progress-2.0.3.tgz",
@ -1702,9 +1711,9 @@
"dev": true "dev": true
}, },
"typescript": { "typescript": {
"version": "3.7.3", "version": "3.8.2",
"resolved": "https://registry.npmjs.org/typescript/-/typescript-3.7.3.tgz", "resolved": "https://registry.npmjs.org/typescript/-/typescript-3.8.2.tgz",
"integrity": "sha512-Mcr/Qk7hXqFBXMN7p7Lusj1ktCBydylfQM/FZCk5glCNQJrCUKPkMHdo9R0MTFWsC/4kPFvDS0fDPvukfCkFsw==", "integrity": "sha512-EgOVgL/4xfVrCMbhYKUQTdF37SQn4Iw73H5BgCrF1Abdun7Kwy/QZsE/ssAy0y4LxBbvua3PIbFsbRczWWnDdQ==",
"dev": true "dev": true
}, },
"uri-js": { "uri-js": {

11
package.json
View File

@ -24,14 +24,15 @@
"package": "npm run install:all && npm run clean && npm run build:serverProd && npm run build:cli && cd client && npx vsce package && cd .." "package": "npm run install:all && npm run clean && npm run build:serverProd && npm run build:cli && cd client && npx vsce package && cd .."
}, },
"devDependencies": { "devDependencies": {
"@types/mocha": "^5.2.7", "@types/node": "^12.12.28",
"@types/node": "^12.12.21", "@typescript-eslint/eslint-plugin": "^2.21.0",
"@typescript-eslint/eslint-plugin": "^2.12.0", "@typescript-eslint/parser": "^2.21.0",
"@typescript-eslint/parser": "^2.12.0",
"del-cli": "^3.0.0", "del-cli": "^3.0.0",
"eslint": "^6.8.0", "eslint": "^6.8.0",
"eslint-config-prettier": "^6.10.0",
"eslint-plugin-simple-import-sort": "^5.0.1", "eslint-plugin-simple-import-sort": "^5.0.1",
"typescript": "^3.7.3" "prettier": "1.19.1",
"typescript": "^3.8.2"
}, },
"main": "index.js", "main": "index.js",
"bin": { "bin": {

1
server/.eslintignore
View File

@ -1 +1,2 @@
**/client/server/**
**/tests/fourslash/** **/tests/fourslash/**

4
server/.vscode/settings.json vendored
View File

@ -1,4 +0,0 @@
{
"javascript.format.insertSpaceAfterOpeningAndBeforeClosingTemplateStringBraces": true,
"typescript.format.insertSpaceAfterOpeningAndBeforeClosingTemplateStringBraces": true
}

2
server/copyTypeshedFallback.js
View File

@ -1,4 +1,5 @@
#!/usr/bin/env node #!/usr/bin/env node
/* eslint-disable @typescript-eslint/no-var-requires */
// This script helps build the command-line version of pyright // This script helps build the command-line version of pyright
// by copying the typeshed-fallback directory to the dist directory. // by copying the typeshed-fallback directory to the dist directory.
@ -10,4 +11,3 @@ fsExtra.emptyDirSync('../dist');
fsExtra.mkdirSync('../dist/typeshed-fallback'); fsExtra.mkdirSync('../dist/typeshed-fallback');
fsExtra.copySync('../client/typeshed-fallback', '../dist/typeshed-fallback'); fsExtra.copySync('../client/typeshed-fallback', '../dist/typeshed-fallback');

59
server/customInstallServerIntoExtension.js
View File

@ -1,55 +1,68 @@
#!/usr/bin/env node #!/usr/bin/env node
/* eslint-disable @typescript-eslint/no-var-requires */
// This file is based on the "installServerIntoExtension" that ships with the // This file is based on the "installServerIntoExtension" that ships with the
// vscode-languagserver node package. We needed to modify it because the original // vscode-languagserver node package. We needed to modify it because the original
// version does not copy the package-lock.json file, and it uses npm update // version does not copy the package-lock.json file, and it uses npm update
// rather than npm install. // rather than npm install.
var path = require('path'); const path = require('path');
var fs = require('fs'); const fs = require('fs');
var cp = require('child_process'); const cp = require('child_process');
var extensionDirectory = process.argv[2]; let extensionDirectory = process.argv[2];
if (!extensionDirectory) { if (!extensionDirectory) {
console.error('No extension directory provided.'); console.error('No extension directory provided.');
process.exit(1) process.exit(1);
} }
extensionDirectory = path.resolve(extensionDirectory) extensionDirectory = path.resolve(extensionDirectory);
if (!fs.existsSync(extensionDirectory)) { if (!fs.existsSync(extensionDirectory)) {
console.error('Extension directory ' + extensionDirectory + ' doesn\'t exist on disk.'); console.error('Extension directory ' + extensionDirectory + " doesn't exist on disk.");
process.exit(1); process.exit(1);
} }
var packageFile = process.argv[3]; let packageFile = process.argv[3];
if (!packageFile) { if (!packageFile) {
console.error('No package.json file provided.'); console.error('No package.json file provided.');
process.exit(1); process.exit(1);
} }
packageFile = path.resolve(packageFile); packageFile = path.resolve(packageFile);
if (!fs.existsSync(packageFile)) { if (!fs.existsSync(packageFile)) {
console.error('Package file ' + packageFile + ' doesn\'t exist on disk.'); console.error('Package file ' + packageFile + " doesn't exist on disk.");
process.exit(1); process.exit(1);
} }
var tsconfigFile = process.argv[4]; let tsconfigFile = process.argv[4];
if (!tsconfigFile) { if (!tsconfigFile) {
console.error('No tsconfig.json file provided'); console.error('No tsconfig.json file provided');
process.exit(1); process.exit(1);
} }
tsconfigFile = path.resolve(tsconfigFile); tsconfigFile = path.resolve(tsconfigFile);
if (!fs.existsSync(tsconfigFile)) { if (!fs.existsSync(tsconfigFile)) {
console.error('tsconfig file ' + tsconfigFile + ' doesn\'t exist on disk.') console.error('tsconfig file ' + tsconfigFile + " doesn't exist on disk.");
process.exit(1); process.exit(1);
} }
var extensionServerDirectory = path.join(extensionDirectory, 'server') const extensionServerDirectory = path.join(extensionDirectory, 'server');
var json = require(tsconfigFile); const json = require(tsconfigFile);
var compilerOptions = json.compilerOptions; const compilerOptions = json.compilerOptions;
if (compilerOptions) { if (compilerOptions) {
var outDir = compilerOptions.outDir; const outDir = compilerOptions.outDir;
if (!outDir || path.join(path.dirname(tsconfigFile), outDir) !== extensionServerDirectory) { if (!outDir || path.join(path.dirname(tsconfigFile), outDir) !== extensionServerDirectory) {
console.error('outDir in ' + process.argv[4] + ' must point to ' + extensionServerDirectory + ' but it points to ' + path.join(path.dirname(tsconfigFile), outDir)); console.error(
console.error('Please change outDir in ' + process.argv[4] + ' to ' + path.relative(path.dirname(tsconfigFile), extensionServerDirectory).replace(/\\/g, '/')); 'outDir in ' +
process.argv[4] +
' must point to ' +
extensionServerDirectory +
' but it points to ' +
path.join(path.dirname(tsconfigFile), outDir)
);
console.error(
'Please change outDir in ' +
process.argv[4] +
' to ' +
path.relative(path.dirname(tsconfigFile), extensionServerDirectory).replace(/\\/g, '/')
);
process.exit(1); process.exit(1);
} }
} }
@ -58,18 +71,18 @@ if (!fs.existsSync(extensionServerDirectory)) {
fs.mkdirSync(extensionServerDirectory); fs.mkdirSync(extensionServerDirectory);
} }
var dest = path.join(extensionServerDirectory, 'package.json'); const dest = path.join(extensionServerDirectory, 'package.json');
console.log('Copying package.json to extension\'s server location...'); console.log("Copying package.json to extension's server location...");
fs.writeFileSync(dest, fs.readFileSync(packageFile)); fs.writeFileSync(dest, fs.readFileSync(packageFile));
var packageLockFile = process.argv[5]; let packageLockFile = process.argv[5];
if (fs.existsSync(packageLockFile)) { if (fs.existsSync(packageLockFile)) {
const packageLockFileDest = path.join(extensionServerDirectory, 'package-lock.json'); const packageLockFileDest = path.join(extensionServerDirectory, 'package-lock.json');
packageLockFile = path.resolve(packageLockFile); packageLockFile = path.resolve(packageLockFile);
console.log('Copying package-lock.json to extension\'s server location...'); console.log("Copying package-lock.json to extension's server location...");
fs.writeFileSync(packageLockFileDest, fs.readFileSync(packageLockFile)); fs.writeFileSync(packageLockFileDest, fs.readFileSync(packageLockFile));
} }
console.log('Installing server npm modules into extension\'s server location...'); console.log("Installing server npm modules into extension's server location...");
process.chdir(extensionServerDirectory) process.chdir(extensionServerDirectory);
cp.execSync('npm install --production --prefix'); cp.execSync('npm install --production --prefix');

18
server/jest.config.js
View File

@ -5,29 +5,21 @@
*/ */
module.exports = { module.exports = {
roots: [ roots: ['<rootDir>/src/tests'],
'<rootDir>/src/tests'
],
transform: { transform: {
'^.+\\.tsx?$': 'ts-jest' '^.+\\.tsx?$': 'ts-jest'
}, },
testRegex: '(/__tests__/.*|(\\.|/)(test|spec))\\.tsx?$', testRegex: '(/__tests__/.*|(\\.|/)(test|spec))\\.tsx?$',
moduleFileExtensions: [ moduleFileExtensions: ['ts', 'tsx', 'js', 'jsx'],
'ts',
'tsx',
'js',
'jsx'
],
globals: { globals: {
'ts-jest': { 'ts-jest': {
tsConfig: { tsConfig: {
"target": "es6", target: 'es6',
// Needed because jest calls tsc in a way that doesn't // Needed because jest calls tsc in a way that doesn't
// inline const enums. // inline const enums.
"preserveConstEnums": true preserveConstEnums: true
} }
} }
} }
} };

228
server/package-lock.json generated
View File

@ -456,9 +456,9 @@
"dev": true "dev": true
}, },
"@types/fs-extra": { "@types/fs-extra": {
"version": "8.0.1", "version": "8.1.0",
"resolved": "https://registry.npmjs.org/@types/fs-extra/-/fs-extra-8.0.1.tgz", "resolved": "https://registry.npmjs.org/@types/fs-extra/-/fs-extra-8.1.0.tgz",
"integrity": "sha512-J00cVDALmi/hJOYsunyT52Hva5TnJeKP5yd1r+mH/ZU0mbYZflR0Z5kw5kITtKTRYMhm1JMClOFYdHnQszEvqw==", "integrity": "sha512-UoOfVEzAUpeSPmjm7h1uk5MH6KZma2z2O7a75onTGjnNvAvMVrPzPL/vBbT65iIGHWj6rokwfmYcmxmlSf2uwg==",
"dev": true, "dev": true,
"requires": { "requires": {
"@types/node": "*" "@types/node": "*"
@ -490,24 +490,24 @@
} }
}, },
"@types/jest": { "@types/jest": {
"version": "24.0.24", "version": "24.9.1",
"resolved": "https://registry.npmjs.org/@types/jest/-/jest-24.0.24.tgz", "resolved": "https://registry.npmjs.org/@types/jest/-/jest-24.9.1.tgz",
"integrity": "sha512-vgaG968EDPSJPMunEDdZvZgvxYSmeH8wKqBlHSkBt1pV2XlLEVDzsj1ZhLuI4iG4Pv841tES61txSBF0obh4CQ==", "integrity": "sha512-Fb38HkXSVA4L8fGKEZ6le5bB8r6MRWlOCZbVuWZcmOMSCd2wCYOwN1ibj8daIoV9naq7aaOZjrLCoCMptKU/4Q==",
"dev": true, "dev": true,
"requires": { "requires": {
"jest-diff": "^24.3.0" "jest-diff": "^24.3.0"
} }
}, },
"@types/json-schema": { "@types/json-schema": {
"version": "7.0.3", "version": "7.0.4",
"resolved": "https://registry.npmjs.org/@types/json-schema/-/json-schema-7.0.3.tgz", "resolved": "https://registry.npmjs.org/@types/json-schema/-/json-schema-7.0.4.tgz",
"integrity": "sha512-Il2DtDVRGDcqjDtE+rF8iqg1CArehSK84HZJCT7AMITlyXRBpuPhqGLDQMowraqqu1coEaimg4ZOqggt6L6L+A==", "integrity": "sha512-8+KAKzEvSUdeo+kmqnKrqgeE+LcA0tjYWFY7RPProVYwnqDjukzO+3b6dLD56rYX5TdWejnEOLJYOIeh4CXKuA==",
"dev": true "dev": true
}, },
"@types/node": { "@types/node": {
"version": "12.12.21", "version": "12.12.28",
"resolved": "https://registry.npmjs.org/@types/node/-/node-12.12.21.tgz", "resolved": "https://registry.npmjs.org/@types/node/-/node-12.12.28.tgz",
"integrity": "sha512-8sRGhbpU+ck1n0PGAUgVrWrWdjSW2aqNeyC15W88GRsMpSwzv6RJGlLhE7s2RhVSOdyDmxbqlWSeThq4/7xqlA==", "integrity": "sha512-g73GJYJDXgf0jqg+P9S8h2acWbDXNkoCX8DLtJVu7Fkn788pzQ/oJsrdJz/2JejRf/SjfZaAhsw+3nd1D5EWGg==",
"dev": true "dev": true
}, },
"@types/stack-utils": { "@types/stack-utils": {
@ -532,12 +532,12 @@
"dev": true "dev": true
}, },
"@typescript-eslint/eslint-plugin": { "@typescript-eslint/eslint-plugin": {
"version": "2.12.0", "version": "2.21.0",
"resolved": "https://registry.npmjs.org/@typescript-eslint/eslint-plugin/-/eslint-plugin-2.12.0.tgz", "resolved": "https://registry.npmjs.org/@typescript-eslint/eslint-plugin/-/eslint-plugin-2.21.0.tgz",
"integrity": "sha512-1t4r9rpLuEwl3hgt90jY18wJHSyb0E3orVL3DaqwmpiSDHmHiSspVsvsFF78BJ/3NNG3qmeso836jpuBWYziAA==", "integrity": "sha512-b5jjjDMxzcjh/Sbjuo7WyhrQmVJg0WipTHQgXh5Xwx10uYm6nPWqN1WGOsaNq4HR3Zh4wUx4IRQdDkCHwyewyw==",
"dev": true, "dev": true,
"requires": { "requires": {
"@typescript-eslint/experimental-utils": "2.12.0", "@typescript-eslint/experimental-utils": "2.21.0",
"eslint-utils": "^1.4.3", "eslint-utils": "^1.4.3",
"functional-red-black-tree": "^1.0.1", "functional-red-black-tree": "^1.0.1",
"regexpp": "^3.0.0", "regexpp": "^3.0.0",
@ -545,51 +545,39 @@
} }
}, },
"@typescript-eslint/experimental-utils": { "@typescript-eslint/experimental-utils": {
"version": "2.12.0", "version": "2.21.0",
"resolved": "https://registry.npmjs.org/@typescript-eslint/experimental-utils/-/experimental-utils-2.12.0.tgz", "resolved": "https://registry.npmjs.org/@typescript-eslint/experimental-utils/-/experimental-utils-2.21.0.tgz",
"integrity": "sha512-jv4gYpw5N5BrWF3ntROvCuLe1IjRenLy5+U57J24NbPGwZFAjhnM45qpq0nDH1y/AZMb3Br25YiNVwyPbz6RkA==", "integrity": "sha512-olKw9JP/XUkav4lq0I7S1mhGgONJF9rHNhKFn9wJlpfRVjNo3PPjSvybxEldvCXnvD+WAshSzqH5cEjPp9CsBA==",
"dev": true, "dev": true,
"requires": { "requires": {
"@types/json-schema": "^7.0.3", "@types/json-schema": "^7.0.3",
"@typescript-eslint/typescript-estree": "2.12.0", "@typescript-eslint/typescript-estree": "2.21.0",
"eslint-scope": "^5.0.0" "eslint-scope": "^5.0.0"
},
"dependencies": {
"eslint-scope": {
"version": "5.0.0",
"resolved": "https://registry.npmjs.org/eslint-scope/-/eslint-scope-5.0.0.tgz",
"integrity": "sha512-oYrhJW7S0bxAFDvWqzvMPRm6pcgcnWc4QnofCAqRTRfQC0JcwenzGglTtsLyIuuWFfkqDG9vz67cnttSd53djw==",
"dev": true,
"requires": {
"esrecurse": "^4.1.0",
"estraverse": "^4.1.1"
}
}
} }
}, },
"@typescript-eslint/parser": { "@typescript-eslint/parser": {
"version": "2.12.0", "version": "2.21.0",
"resolved": "https://registry.npmjs.org/@typescript-eslint/parser/-/parser-2.12.0.tgz", "resolved": "https://registry.npmjs.org/@typescript-eslint/parser/-/parser-2.21.0.tgz",
"integrity": "sha512-lPdkwpdzxEfjI8TyTzZqPatkrswLSVu4bqUgnB03fHSOwpC7KSerPgJRgIAf11UGNf7HKjJV6oaPZI4AghLU6g==", "integrity": "sha512-VrmbdrrrvvI6cPPOG7uOgGUFXNYTiSbnRq8ZMyuGa4+qmXJXVLEEz78hKuqupvkpwJQNk1Ucz1TenrRP90gmBg==",
"dev": true, "dev": true,
"requires": { "requires": {
"@types/eslint-visitor-keys": "^1.0.0", "@types/eslint-visitor-keys": "^1.0.0",
"@typescript-eslint/experimental-utils": "2.12.0", "@typescript-eslint/experimental-utils": "2.21.0",
"@typescript-eslint/typescript-estree": "2.12.0", "@typescript-eslint/typescript-estree": "2.21.0",
"eslint-visitor-keys": "^1.1.0" "eslint-visitor-keys": "^1.1.0"
} }
}, },
"@typescript-eslint/typescript-estree": { "@typescript-eslint/typescript-estree": {
"version": "2.12.0", "version": "2.21.0",
"resolved": "https://registry.npmjs.org/@typescript-eslint/typescript-estree/-/typescript-estree-2.12.0.tgz", "resolved": "https://registry.npmjs.org/@typescript-eslint/typescript-estree/-/typescript-estree-2.21.0.tgz",
"integrity": "sha512-rGehVfjHEn8Frh9UW02ZZIfJs6SIIxIu/K1bbci8rFfDE/1lQ8krIJy5OXOV3DVnNdDPtoiPOdEANkLMrwXbiQ==", "integrity": "sha512-NC/nogZNb9IK2MEFQqyDBAciOT8Lp8O3KgAfvHx2Skx6WBo+KmDqlU3R9KxHONaijfTIKtojRe3SZQyMjr3wBw==",
"dev": true, "dev": true,
"requires": { "requires": {
"debug": "^4.1.1", "debug": "^4.1.1",
"eslint-visitor-keys": "^1.1.0", "eslint-visitor-keys": "^1.1.0",
"glob": "^7.1.6", "glob": "^7.1.6",
"is-glob": "^4.0.1", "is-glob": "^4.0.1",
"lodash.unescape": "4.0.1", "lodash": "^4.17.15",
"semver": "^6.3.0", "semver": "^6.3.0",
"tsutils": "^3.17.1" "tsutils": "^3.17.1"
}, },
@ -1531,9 +1519,9 @@
} }
}, },
"chownr": { "chownr": {
"version": "1.1.3", "version": "1.1.4",
"resolved": "https://registry.npmjs.org/chownr/-/chownr-1.1.3.tgz", "resolved": "https://registry.npmjs.org/chownr/-/chownr-1.1.4.tgz",
"integrity": "sha512-i70fVHhmV3DtTl6nqvZOnIjbY0Pe4kAUjwHj8z0zAdgBtYrJyYwLKCCuRBQ5ppkyL0AkN7HKRnETdmdp1zqNXw==", "integrity": "sha512-jJ0bqzaylmJtVnNgzTeSOs8DPavpbYgEr/b0YL8/2GO3xJEhInFmhKMUnEJQjZumK7KXGFhUy89PrsJWlakBVg==",
"dev": true "dev": true
}, },
"chrome-trace-event": { "chrome-trace-event": {
@ -1660,9 +1648,9 @@
} }
}, },
"commander": { "commander": {
"version": "2.20.1", "version": "2.20.3",
"resolved": "https://registry.npmjs.org/commander/-/commander-2.20.1.tgz", "resolved": "https://registry.npmjs.org/commander/-/commander-2.20.3.tgz",
"integrity": "sha512-cCuLsMhJeWQ/ZpsFTbE765kvVfoeSddc4nU3up4fV+fDBcfUXnbITJ+JzhkdjzOqhURjZgujxaioam4RM9yGUg==", "integrity": "sha512-GpVkmM8vF2vQUkj2LvZmD35JxeJOLCwJ9cUkugyk2nuhbv3+mJvpLYYt+0+USMxE+oj+ey/lJEnhZw75x/OMcQ==",
"dev": true "dev": true
}, },
"commondir": { "commondir": {
@ -2109,6 +2097,7 @@
"version": "1.14.2", "version": "1.14.2",
"resolved": "https://registry.npmjs.org/es-abstract/-/es-abstract-1.14.2.tgz", "resolved": "https://registry.npmjs.org/es-abstract/-/es-abstract-1.14.2.tgz",
"integrity": "sha512-DgoQmbpFNOofkjJtKwr87Ma5EW4Dc8fWhD0R+ndq7Oc456ivUfGOOP6oAZTTKl5/CcNMP+EN+e3/iUzgE0veZg==", "integrity": "sha512-DgoQmbpFNOofkjJtKwr87Ma5EW4Dc8fWhD0R+ndq7Oc456ivUfGOOP6oAZTTKl5/CcNMP+EN+e3/iUzgE0veZg==",
"dev": true,
"requires": { "requires": {
"es-to-primitive": "^1.2.0", "es-to-primitive": "^1.2.0",
"function-bind": "^1.1.1", "function-bind": "^1.1.1",
@ -2126,6 +2115,7 @@
"version": "1.2.0", "version": "1.2.0",
"resolved": "https://registry.npmjs.org/es-to-primitive/-/es-to-primitive-1.2.0.tgz", "resolved": "https://registry.npmjs.org/es-to-primitive/-/es-to-primitive-1.2.0.tgz",
"integrity": "sha512-qZryBOJjV//LaxLTV6UC//WewneB3LcXOL9NP++ozKVXsIIIpm/2c13UDiD9Jp2eThsecw9m3jPqDwTyobcdbg==", "integrity": "sha512-qZryBOJjV//LaxLTV6UC//WewneB3LcXOL9NP++ozKVXsIIIpm/2c13UDiD9Jp2eThsecw9m3jPqDwTyobcdbg==",
"dev": true,
"requires": { "requires": {
"is-callable": "^1.1.4", "is-callable": "^1.1.4",
"is-date-object": "^1.0.1", "is-date-object": "^1.0.1",
@ -2263,9 +2253,9 @@
} }
}, },
"eslint-scope": { "eslint-scope": {
"version": "4.0.3", "version": "5.0.0",
"resolved": "https://registry.npmjs.org/eslint-scope/-/eslint-scope-4.0.3.tgz", "resolved": "https://registry.npmjs.org/eslint-scope/-/eslint-scope-5.0.0.tgz",
"integrity": "sha512-p7VutNr1O/QrxysMo3E45FjYDTeXBy0iTltPFNSqKAIfjDSXC+4dj+qfyuD8bfAXrW/y6lW3O76VaYNPKfpKrg==", "integrity": "sha512-oYrhJW7S0bxAFDvWqzvMPRm6pcgcnWc4QnofCAqRTRfQC0JcwenzGglTtsLyIuuWFfkqDG9vz67cnttSd53djw==",
"dev": true, "dev": true,
"requires": { "requires": {
"esrecurse": "^4.1.0", "esrecurse": "^4.1.0",
@ -2343,9 +2333,9 @@
"dev": true "dev": true
}, },
"events": { "events": {
"version": "3.0.0", "version": "3.1.0",
"resolved": "https://registry.npmjs.org/events/-/events-3.0.0.tgz", "resolved": "https://registry.npmjs.org/events/-/events-3.1.0.tgz",
"integrity": "sha512-Dc381HFWJzEOhQ+d8pkNon++bk9h6cdAoAj4iE6Q4y6xgTzySWXlKn05/TVNpjnfRqi/X0EpJEJohPjNI3zpVA==", "integrity": "sha512-Rv+u8MLHNOdMjTAFeT3nCjHn2aGlx435FP/sDHNaRhDEMwyI/aB22Kj2qIN8R0cw3z28psEQLYwxVKLsKrMgWg==",
"dev": true "dev": true
}, },
"evp_bytestokey": { "evp_bytestokey": {
@ -3342,7 +3332,8 @@
"function-bind": { "function-bind": {
"version": "1.1.1", "version": "1.1.1",
"resolved": "https://registry.npmjs.org/function-bind/-/function-bind-1.1.1.tgz", "resolved": "https://registry.npmjs.org/function-bind/-/function-bind-1.1.1.tgz",
"integrity": "sha512-yIovAzMX49sF8Yl58fSCWJ5svSLuaibPxXQJFLmBObTuCr0Mf1KiPopGM9NiFjiYBCbfaa2Fh6breQ6ANVTI0A==" "integrity": "sha512-yIovAzMX49sF8Yl58fSCWJ5svSLuaibPxXQJFLmBObTuCr0Mf1KiPopGM9NiFjiYBCbfaa2Fh6breQ6ANVTI0A==",
"dev": true
}, },
"functional-red-black-tree": { "functional-red-black-tree": {
"version": "1.0.1", "version": "1.0.1",
@ -3487,6 +3478,7 @@
"version": "1.0.3", "version": "1.0.3",
"resolved": "https://registry.npmjs.org/has/-/has-1.0.3.tgz", "resolved": "https://registry.npmjs.org/has/-/has-1.0.3.tgz",
"integrity": "sha512-f2dvO0VU6Oej7RkWJGrehjbzMAjFp5/VKPp5tTpWIV4JHHZK1/BxbFRtf/siA2SWTe09caDmVtYYzWEIbBS4zw==", "integrity": "sha512-f2dvO0VU6Oej7RkWJGrehjbzMAjFp5/VKPp5tTpWIV4JHHZK1/BxbFRtf/siA2SWTe09caDmVtYYzWEIbBS4zw==",
"dev": true,
"requires": { "requires": {
"function-bind": "^1.1.1" "function-bind": "^1.1.1"
} }
@ -3499,7 +3491,8 @@
"has-symbols": { "has-symbols": {
"version": "1.0.0", "version": "1.0.0",
"resolved": "https://registry.npmjs.org/has-symbols/-/has-symbols-1.0.0.tgz", "resolved": "https://registry.npmjs.org/has-symbols/-/has-symbols-1.0.0.tgz",
"integrity": "sha1-uhqPGvKg/DllD1yFA2dwQSIGO0Q=" "integrity": "sha1-uhqPGvKg/DllD1yFA2dwQSIGO0Q=",
"dev": true
}, },
"has-value": { "has-value": {
"version": "1.0.0", "version": "1.0.0",
@ -3843,7 +3836,8 @@
"is-callable": { "is-callable": {
"version": "1.1.4", "version": "1.1.4",
"resolved": "https://registry.npmjs.org/is-callable/-/is-callable-1.1.4.tgz", "resolved": "https://registry.npmjs.org/is-callable/-/is-callable-1.1.4.tgz",
"integrity": "sha512-r5p9sxJjYnArLjObpjA4xu5EKI3CuKHkJXMhT7kwbpUyIFD1n5PMAsoPvWnvtZiNz7LjkYDRZhd7FlI0eMijEA==" "integrity": "sha512-r5p9sxJjYnArLjObpjA4xu5EKI3CuKHkJXMhT7kwbpUyIFD1n5PMAsoPvWnvtZiNz7LjkYDRZhd7FlI0eMijEA==",
"dev": true
}, },
"is-ci": { "is-ci": {
"version": "2.0.0", "version": "2.0.0",
@ -3877,7 +3871,8 @@
"is-date-object": { "is-date-object": {
"version": "1.0.1", "version": "1.0.1",
"resolved": "https://registry.npmjs.org/is-date-object/-/is-date-object-1.0.1.tgz", "resolved": "https://registry.npmjs.org/is-date-object/-/is-date-object-1.0.1.tgz",
"integrity": "sha1-mqIOtq7rv/d/vTPnTKAbM1gdOhY=" "integrity": "sha1-mqIOtq7rv/d/vTPnTKAbM1gdOhY=",
"dev": true
}, },
"is-descriptor": { "is-descriptor": {
"version": "0.1.6", "version": "0.1.6",
@ -3935,11 +3930,11 @@
} }
}, },
"is-nan": { "is-nan": {
"version": "1.2.1", "version": "1.3.0",
"resolved": "https://registry.npmjs.org/is-nan/-/is-nan-1.2.1.tgz", "resolved": "https://registry.npmjs.org/is-nan/-/is-nan-1.3.0.tgz",
"integrity": "sha1-n69ltvttskt/XAYoR16nH5iEAeI=", "integrity": "sha512-z7bbREymOqt2CCaZVly8aC4ML3Xhfi0ekuOnjO2L8vKdl+CttdVoGZQhd4adMFAsxQ5VeRVwORs4tU8RH+HFtQ==",
"requires": { "requires": {
"define-properties": "^1.1.1" "define-properties": "^1.1.3"
} }
}, },
"is-number": { "is-number": {
@ -3981,6 +3976,7 @@
"version": "1.0.4", "version": "1.0.4",
"resolved": "https://registry.npmjs.org/is-regex/-/is-regex-1.0.4.tgz", "resolved": "https://registry.npmjs.org/is-regex/-/is-regex-1.0.4.tgz",
"integrity": "sha1-VRdIm1RwkbCTDglWVM7SXul+lJE=", "integrity": "sha1-VRdIm1RwkbCTDglWVM7SXul+lJE=",
"dev": true,
"requires": { "requires": {
"has": "^1.0.1" "has": "^1.0.1"
} }
@ -3995,6 +3991,7 @@
"version": "1.0.2", "version": "1.0.2",
"resolved": "https://registry.npmjs.org/is-symbol/-/is-symbol-1.0.2.tgz", "resolved": "https://registry.npmjs.org/is-symbol/-/is-symbol-1.0.2.tgz",
"integrity": "sha512-HS8bZ9ox60yCJLH9snBpIwv9pYUAkcuLhSA1oero1UB5y9aiQpRA8y2ex945AOtCZL1lJDeIk3G5LthswI46Lw==", "integrity": "sha512-HS8bZ9ox60yCJLH9snBpIwv9pYUAkcuLhSA1oero1UB5y9aiQpRA8y2ex945AOtCZL1lJDeIk3G5LthswI46Lw==",
"dev": true,
"requires": { "requires": {
"has-symbols": "^1.0.0" "has-symbols": "^1.0.0"
} }
@ -4802,12 +4799,6 @@
"integrity": "sha1-7dFMgk4sycHgsKG0K7UhBRakJDg=", "integrity": "sha1-7dFMgk4sycHgsKG0K7UhBRakJDg=",
"dev": true "dev": true
}, },
"lodash.unescape": {
"version": "4.0.1",
"resolved": "https://registry.npmjs.org/lodash.unescape/-/lodash.unescape-4.0.1.tgz",
"integrity": "sha1-vyJJiGzlFM2hEvrpIYzcBlIR/Jw=",
"dev": true
},
"loose-envify": { "loose-envify": {
"version": "1.4.0", "version": "1.4.0",
"resolved": "https://registry.npmjs.org/loose-envify/-/loose-envify-1.4.0.tgz", "resolved": "https://registry.npmjs.org/loose-envify/-/loose-envify-1.4.0.tgz",
@ -4845,9 +4836,9 @@
} }
}, },
"make-error": { "make-error": {
"version": "1.3.5", "version": "1.3.6",
"resolved": "https://registry.npmjs.org/make-error/-/make-error-1.3.5.tgz", "resolved": "https://registry.npmjs.org/make-error/-/make-error-1.3.6.tgz",
"integrity": "sha512-c3sIjNUow0+8swNwVpqoH4YCShKNFkMaw6oH1mNS2haDZQqkeZFlHS3dhoeEbKKmJB4vXpJucU6oH75aDYeE9g==", "integrity": "sha512-s8UhlNe7vPKomQhC1qFelMokr/Sc3AgNbso3n74mVPA5LTZwkB9NlXf4XPamLxJE8h0gh73rM94xvwRT2CVInw==",
"dev": true "dev": true
}, },
"makeerror": { "makeerror": {
@ -5322,12 +5313,13 @@
"object-inspect": { "object-inspect": {
"version": "1.6.0", "version": "1.6.0",
"resolved": "https://registry.npmjs.org/object-inspect/-/object-inspect-1.6.0.tgz", "resolved": "https://registry.npmjs.org/object-inspect/-/object-inspect-1.6.0.tgz",
"integrity": "sha512-GJzfBZ6DgDAmnuaM3104jR4s1Myxr3Y3zfIyN4z3UdqN69oSRacNK8UhnobDdC+7J2AHCjGwxQubNJfE70SXXQ==" "integrity": "sha512-GJzfBZ6DgDAmnuaM3104jR4s1Myxr3Y3zfIyN4z3UdqN69oSRacNK8UhnobDdC+7J2AHCjGwxQubNJfE70SXXQ==",
"dev": true
}, },
"object-is": { "object-is": {
"version": "1.0.1", "version": "1.0.2",
"resolved": "https://registry.npmjs.org/object-is/-/object-is-1.0.1.tgz", "resolved": "https://registry.npmjs.org/object-is/-/object-is-1.0.2.tgz",
"integrity": "sha1-CqYOyZiaCz7Xlc9NBvYs8a1lObY=" "integrity": "sha512-Epah+btZd5wrrfjkJZq1AOB9O6OxUQto45hzFd7lXGrpHPGE0W1k+426yrZV+k6NJOzLNNW/nVsmZdIWsAqoOQ=="
}, },
"object-keys": { "object-keys": {
"version": "1.1.1", "version": "1.1.1",
@ -5343,17 +5335,6 @@
"isobject": "^3.0.0" "isobject": "^3.0.0"
} }
}, },
"object.entries": {
"version": "1.1.0",
"resolved": "https://registry.npmjs.org/object.entries/-/object.entries-1.1.0.tgz",
"integrity": "sha512-l+H6EQ8qzGRxbkHOd5I/aHRhHDKoQXQ8g0BYt4uSweQU1/J6dZUOyWh9a2Vky35YCKjzmgxOzta2hH6kf9HuXA==",
"requires": {
"define-properties": "^1.1.3",
"es-abstract": "^1.12.0",
"function-bind": "^1.1.1",
"has": "^1.0.3"
}
},
"object.getownpropertydescriptors": { "object.getownpropertydescriptors": {
"version": "2.0.3", "version": "2.0.3",
"resolved": "https://registry.npmjs.org/object.getownpropertydescriptors/-/object.getownpropertydescriptors-2.0.3.tgz", "resolved": "https://registry.npmjs.org/object.getownpropertydescriptors/-/object.getownpropertydescriptors-2.0.3.tgz",
@ -5512,9 +5493,9 @@
"dev": true "dev": true
}, },
"pako": { "pako": {
"version": "1.0.10", "version": "1.0.11",
"resolved": "https://registry.npmjs.org/pako/-/pako-1.0.10.tgz", "resolved": "https://registry.npmjs.org/pako/-/pako-1.0.11.tgz",
"integrity": "sha512-0DTvPVU3ed8+HNXOu5Bs+o//Mbdj9VNQMUOe9oKCwh8l0GNwpTDMKCWbRjgtD291AWnkAgkqA/LOnQS8AmS1tw==", "integrity": "sha512-4hLB8Py4zZce5s4yd9XzopqwVv/yGNhV1Bl8NTmCq1763HeK2+EwVTv+leGeL13Dnh2wfbqowVPXCIO0z4taYw==",
"dev": true "dev": true
}, },
"parallel-transform": { "parallel-transform": {
@ -6654,6 +6635,7 @@
"version": "2.1.0", "version": "2.1.0",
"resolved": "https://registry.npmjs.org/string.prototype.trimleft/-/string.prototype.trimleft-2.1.0.tgz", "resolved": "https://registry.npmjs.org/string.prototype.trimleft/-/string.prototype.trimleft-2.1.0.tgz",
"integrity": "sha512-FJ6b7EgdKxxbDxc79cOlok6Afd++TTs5szo+zJTUyow3ycrRfJVE2pq3vcN53XexvKZu/DJMDfeI/qMiZTrjTw==", "integrity": "sha512-FJ6b7EgdKxxbDxc79cOlok6Afd++TTs5szo+zJTUyow3ycrRfJVE2pq3vcN53XexvKZu/DJMDfeI/qMiZTrjTw==",
"dev": true,
"requires": { "requires": {
"define-properties": "^1.1.3", "define-properties": "^1.1.3",
"function-bind": "^1.1.1" "function-bind": "^1.1.1"
@ -6663,6 +6645,7 @@
"version": "2.1.0", "version": "2.1.0",
"resolved": "https://registry.npmjs.org/string.prototype.trimright/-/string.prototype.trimright-2.1.0.tgz", "resolved": "https://registry.npmjs.org/string.prototype.trimright/-/string.prototype.trimright-2.1.0.tgz",
"integrity": "sha512-fXZTSV55dNBwv16uw+hh5jkghxSnc5oHq+5K/gXgizHwAvMetdAJlHqqoFC1FSDVPYWLkAKl2cxpUT41sV7nSg==", "integrity": "sha512-fXZTSV55dNBwv16uw+hh5jkghxSnc5oHq+5K/gXgizHwAvMetdAJlHqqoFC1FSDVPYWLkAKl2cxpUT41sV7nSg==",
"dev": true,
"requires": { "requires": {
"define-properties": "^1.1.3", "define-properties": "^1.1.3",
"function-bind": "^1.1.1" "function-bind": "^1.1.1"
@ -6737,9 +6720,9 @@
"dev": true "dev": true
}, },
"terser": { "terser": {
"version": "4.4.3", "version": "4.6.3",
"resolved": "https://registry.npmjs.org/terser/-/terser-4.4.3.tgz", "resolved": "https://registry.npmjs.org/terser/-/terser-4.6.3.tgz",
"integrity": "sha512-0ikKraVtRDKGzHrzkCv5rUNDzqlhmhowOBqC0XqUHFpW+vJ45+20/IFBcebwKfiS2Z9fJin6Eo+F1zLZsxi8RA==", "integrity": "sha512-Lw+ieAXmY69d09IIc/yqeBqXpEQIpDGZqT34ui1QWXIUpR2RjbqEkT8X7Lgex19hslSqcWM5iMN2kM11eMsESQ==",
"dev": true, "dev": true,
"requires": { "requires": {
"commander": "^2.20.0", "commander": "^2.20.0",
@ -6902,9 +6885,9 @@
} }
}, },
"ts-jest": { "ts-jest": {
"version": "24.2.0", "version": "24.3.0",
"resolved": "https://registry.npmjs.org/ts-jest/-/ts-jest-24.2.0.tgz", "resolved": "https://registry.npmjs.org/ts-jest/-/ts-jest-24.3.0.tgz",
"integrity": "sha512-Yc+HLyldlIC9iIK8xEN7tV960Or56N49MDP7hubCZUeI7EbIOTsas6rXCMB4kQjLACJ7eDOF4xWEO5qumpKsag==", "integrity": "sha512-Hb94C/+QRIgjVZlJyiWwouYUF+siNJHJHknyspaOcZ+OQAIdFG/UrdQVXw/0B8Z3No34xkUXZJpOTy9alOWdVQ==",
"dev": true, "dev": true,
"requires": { "requires": {
"bs-logger": "0.x", "bs-logger": "0.x",
@ -7058,9 +7041,9 @@
"dev": true "dev": true
}, },
"typescript": { "typescript": {
"version": "3.7.3", "version": "3.8.2",
"resolved": "https://registry.npmjs.org/typescript/-/typescript-3.7.3.tgz", "resolved": "https://registry.npmjs.org/typescript/-/typescript-3.8.2.tgz",
"integrity": "sha512-Mcr/Qk7hXqFBXMN7p7Lusj1ktCBydylfQM/FZCk5glCNQJrCUKPkMHdo9R0MTFWsC/4kPFvDS0fDPvukfCkFsw==", "integrity": "sha512-EgOVgL/4xfVrCMbhYKUQTdF37SQn4Iw73H5BgCrF1Abdun7Kwy/QZsE/ssAy0y4LxBbvua3PIbFsbRczWWnDdQ==",
"dev": true "dev": true
}, },
"typescript-char": { "typescript-char": {
@ -7215,14 +7198,13 @@
"dev": true "dev": true
}, },
"util": { "util": {
"version": "0.12.1", "version": "0.12.2",
"resolved": "https://registry.npmjs.org/util/-/util-0.12.1.tgz", "resolved": "https://registry.npmjs.org/util/-/util-0.12.2.tgz",
"integrity": "sha512-MREAtYOp+GTt9/+kwf00IYoHZyjM8VU4aVrkzUlejyqaIjd2GztVl5V9hGXKlvBKE3gENn/FMfHE5v6hElXGcQ==", "integrity": "sha512-XE+MkWQvglYa+IOfBt5UFG93EmncEMP23UqpgDvVZVFBPxwmkK10QRp6pgU4xICPnWRf/t0zPv4noYSUq9gqUQ==",
"requires": { "requires": {
"inherits": "^2.0.3", "inherits": "^2.0.3",
"is-arguments": "^1.0.4", "is-arguments": "^1.0.4",
"is-generator-function": "^1.0.7", "is-generator-function": "^1.0.7",
"object.entries": "^1.1.0",
"safe-buffer": "^5.1.2" "safe-buffer": "^5.1.2"
}, },
"dependencies": { "dependencies": {
@ -7289,9 +7271,9 @@
"dev": true "dev": true
}, },
"vscode-jsonrpc": { "vscode-jsonrpc": {
"version": "5.0.0-next.2", "version": "5.0.1",
"resolved": "https://registry.npmjs.org/vscode-jsonrpc/-/vscode-jsonrpc-5.0.0-next.2.tgz", "resolved": "https://registry.npmjs.org/vscode-jsonrpc/-/vscode-jsonrpc-5.0.1.tgz",
"integrity": "sha512-Q3/jabZUNviCG9hhF6hHWjhrABevPF9mv0aiE2j8BYCAP2k+aHTpjMyk+04MzaAqWYwXdQuZkLSbcYCCqbzJLg==" "integrity": "sha512-JvONPptw3GAQGXlVV2utDcHx0BiY34FupW/kI6mZ5x06ER5DdPG/tXWMVHjTNULF5uKPOUUD0SaXg5QaubJL0A=="
}, },
"vscode-languageserver": { "vscode-languageserver": {
"version": "5.3.0-next.10", "version": "5.3.0-next.10",
@ -7303,18 +7285,18 @@
} }
}, },
"vscode-languageserver-protocol": { "vscode-languageserver-protocol": {
"version": "3.15.0-next.9", "version": "3.15.3",
"resolved": "https://registry.npmjs.org/vscode-languageserver-protocol/-/vscode-languageserver-protocol-3.15.0-next.9.tgz", "resolved": "https://registry.npmjs.org/vscode-languageserver-protocol/-/vscode-languageserver-protocol-3.15.3.tgz",
"integrity": "sha512-b9PAxouMmtsLEe8ZjbIMPb7wRWPhckGfgjwZLmp/dWnaAuRPYtY3lGO0/rNbLc3jKIqCVlnEyYVFKalzDAzj0g==", "integrity": "sha512-zrMuwHOAQRhjDSnflWdJG+O2ztMWss8GqUUB8dXLR/FPenwkiBNkMIJJYfSN6sgskvsF0rHAoBowNQfbyZnnvw==",
"requires": { "requires": {
"vscode-jsonrpc": "^5.0.0-next.2", "vscode-jsonrpc": "^5.0.1",
"vscode-languageserver-types": "^3.15.0-next.5" "vscode-languageserver-types": "3.15.1"
} }
}, },
"vscode-languageserver-types": { "vscode-languageserver-types": {
"version": "3.15.0-next.5", "version": "3.15.1",
"resolved": "https://registry.npmjs.org/vscode-languageserver-types/-/vscode-languageserver-types-3.15.0-next.5.tgz", "resolved": "https://registry.npmjs.org/vscode-languageserver-types/-/vscode-languageserver-types-3.15.1.tgz",
"integrity": "sha512-7hrELhTeWieUgex3+6692KjCkcmO/+V/bFItM5MHGcBotzwmjEuXjapLLYTYhIspuJ1ibRSik5MhX5YwLpsPiw==" "integrity": "sha512-+a9MPUQrNGRrGU630OGbYVQ+11iOIovjCkqxajPa9w57Sd5ruK8WQNsslzpa0x/QJqC8kRc2DUxWjIFwoNm4ZQ=="
}, },
"vscode-textbuffer": { "vscode-textbuffer": {
"version": "1.0.0", "version": "1.0.0",
@ -7443,9 +7425,9 @@
"dev": true "dev": true
}, },
"webpack": { "webpack": {
"version": "4.41.3", "version": "4.41.6",
"resolved": "https://registry.npmjs.org/webpack/-/webpack-4.41.3.tgz", "resolved": "https://registry.npmjs.org/webpack/-/webpack-4.41.6.tgz",
"integrity": "sha512-EcNzP9jGoxpQAXq1VOoTet0ik7/VVU1MovIfcUSAjLowc7GhcQku/sOXALvq5nPpSei2HF6VRhibeJSC3i/Law==", "integrity": "sha512-yxXfV0Zv9WMGRD+QexkZzmGIh54bsvEs+9aRWxnN8erLWEOehAKUTeNBoUbA6HPEZPlRo7KDi2ZcNveoZgK9MA==",
"dev": true, "dev": true,
"requires": { "requires": {
"@webassemblyjs/ast": "1.8.5", "@webassemblyjs/ast": "1.8.5",
@ -7478,13 +7460,23 @@
"resolved": "https://registry.npmjs.org/acorn/-/acorn-6.4.0.tgz", "resolved": "https://registry.npmjs.org/acorn/-/acorn-6.4.0.tgz",
"integrity": "sha512-gac8OEcQ2Li1dxIEWGZzsp2BitJxwkwcOm0zHAJLcPJaVvm58FRnk6RkuLRpU1EujipU2ZFODv2P9DLMfnV8mw==", "integrity": "sha512-gac8OEcQ2Li1dxIEWGZzsp2BitJxwkwcOm0zHAJLcPJaVvm58FRnk6RkuLRpU1EujipU2ZFODv2P9DLMfnV8mw==",
"dev": true "dev": true
},
"eslint-scope": {
"version": "4.0.3",
"resolved": "https://registry.npmjs.org/eslint-scope/-/eslint-scope-4.0.3.tgz",
"integrity": "sha512-p7VutNr1O/QrxysMo3E45FjYDTeXBy0iTltPFNSqKAIfjDSXC+4dj+qfyuD8bfAXrW/y6lW3O76VaYNPKfpKrg==",
"dev": true,
"requires": {
"esrecurse": "^4.1.0",
"estraverse": "^4.1.1"
}
} }
} }
}, },
"webpack-cli": { "webpack-cli": {
"version": "3.3.10", "version": "3.3.11",
"resolved": "https://registry.npmjs.org/webpack-cli/-/webpack-cli-3.3.10.tgz", "resolved": "https://registry.npmjs.org/webpack-cli/-/webpack-cli-3.3.11.tgz",
"integrity": "sha512-u1dgND9+MXaEt74sJR4PR7qkPxXUSQ0RXYq8x1L6Jg1MYVEmGPrH6Ah6C4arD4r0J1P5HKjRqpab36k0eIzPqg==", "integrity": "sha512-dXlfuml7xvAFwYUPsrtQAA9e4DOe58gnzSxhgrO/ZM/gyXTBowrsYeubyN4mqGhYdpXMFNyQ6emjJS9M7OBd4g==",
"dev": true, "dev": true,
"requires": { "requires": {
"chalk": "2.4.2", "chalk": "2.4.2",

18
server/package.json
View File

@ -31,20 +31,20 @@
"@types/chalk": "^2.2.0", "@types/chalk": "^2.2.0",
"@types/chokidar": "^2.1.3", "@types/chokidar": "^2.1.3",
"@types/command-line-args": "^5.0.0", "@types/command-line-args": "^5.0.0",
"@types/fs-extra": "^8.0.1", "@types/fs-extra": "^8.1.0",
"@types/jest": "^24.0.24", "@types/jest": "^24.9.1",
"@types/node": "^12.12.21", "@types/node": "^12.12.28",
"@typescript-eslint/eslint-plugin": "^2.12.0", "@typescript-eslint/eslint-plugin": "^2.21.0",
"@typescript-eslint/parser": "^2.12.0", "@typescript-eslint/parser": "^2.21.0",
"eslint": "^6.8.0", "eslint": "^6.8.0",
"fs-extra": "^8.1.0", "fs-extra": "^8.1.0",
"jest": "^24.9.0", "jest": "^24.9.0",
"node-loader": "^0.6.0", "node-loader": "^0.6.0",
"ts-jest": "^24.2.0", "ts-jest": "^24.3.0",
"ts-loader": "^6.2.1", "ts-loader": "^6.2.1",
"typescript": "^3.7.3", "typescript": "^3.8.2",
"webpack": "^4.41.3", "webpack": "^4.41.6",
"webpack-cli": "^3.3.10" "webpack-cli": "^3.3.11"
}, },
"types": "out/main.d.ts", "types": "out/main.d.ts",
"main": "out/main.js" "main": "out/main.js"

14
server/src/analyzer/analyzerNodeInfo.ts
View File

@ -10,8 +10,15 @@
* (e.g. hover information). * (e.g. hover information).
*/ */
import { ClassNode, ExecutionScopeNode, FunctionNode, LambdaNode, ListComprehensionNode, import {
ModuleNode, ParseNode } from '../parser/parseNodes'; ClassNode,
ExecutionScopeNode,
FunctionNode,
LambdaNode,
ListComprehensionNode,
ModuleNode,
ParseNode
} from '../parser/parseNodes';
import { AnalyzerFileInfo } from './analyzerFileInfo'; import { AnalyzerFileInfo } from './analyzerFileInfo';
import { FlowFlags, FlowNode } from './codeFlow'; import { FlowFlags, FlowNode } from './codeFlow';
import { Declaration } from './declaration'; import { Declaration } from './declaration';
@ -50,8 +57,7 @@ interface AnalyzerNodeInfo {
codeFlowExpressions?: Map<string, string>; codeFlowExpressions?: Map<string, string>;
} }
export type ScopedNode = ModuleNode | ClassNode | FunctionNode | export type ScopedNode = ModuleNode | ClassNode | FunctionNode | LambdaNode | ListComprehensionNode;
LambdaNode | ListComprehensionNode;
// Cleans out all fields that are added by the analyzer phases // Cleans out all fields that are added by the analyzer phases
// (after the post-parse walker). // (after the post-parse walker).

419
server/src/analyzer/binder.ts
View File

@ -25,23 +25,79 @@ import { convertOffsetsToRange } from '../common/positionUtils';
import { PythonVersion } from '../common/pythonVersion'; import { PythonVersion } from '../common/pythonVersion';
import { getEmptyRange } from '../common/textRange'; import { getEmptyRange } from '../common/textRange';
import { TextRange } from '../common/textRange'; import { TextRange } from '../common/textRange';
import { ArgumentCategory, AssertNode, AssignmentExpressionNode, AssignmentNode, import {
AugmentedAssignmentNode, AwaitNode, BinaryOperationNode, BreakNode, ArgumentCategory,
CallNode, ClassNode, ContinueNode, DelNode, ExceptNode, ExpressionNode, ForNode, AssertNode,
FunctionNode, GlobalNode, IfNode, ImportAsNode, ImportFromNode, LambdaNode, AssignmentExpressionNode,
ListComprehensionNode, MemberAccessNode, ModuleNameNode, ModuleNode, NameNode, NonlocalNode, AssignmentNode,
ParseNode, ParseNodeType, RaiseNode, ReturnNode, StatementNode, StringListNode, AugmentedAssignmentNode,
SuiteNode, TernaryNode, TryNode, TypeAnnotationNode, AwaitNode,
UnaryOperationNode, WhileNode, WithNode, YieldFromNode, YieldNode } from '../parser/parseNodes'; BinaryOperationNode,
BreakNode,
CallNode,
ClassNode,
ContinueNode,
DelNode,
ExceptNode,
ExpressionNode,
ForNode,
FunctionNode,
GlobalNode,
IfNode,
ImportAsNode,
ImportFromNode,
LambdaNode,
ListComprehensionNode,
MemberAccessNode,
ModuleNameNode,
ModuleNode,
NameNode,
NonlocalNode,
ParseNode,
ParseNodeType,
RaiseNode,
ReturnNode,
StatementNode,
StringListNode,
SuiteNode,
TernaryNode,
TryNode,
TypeAnnotationNode,
UnaryOperationNode,
WhileNode,
WithNode,
YieldFromNode,
YieldNode
} from '../parser/parseNodes';
import * as StringTokenUtils from '../parser/stringTokenUtils'; import * as StringTokenUtils from '../parser/stringTokenUtils';
import { KeywordType, OperatorType } from '../parser/tokenizerTypes'; import { KeywordType, OperatorType } from '../parser/tokenizerTypes';
import { AnalyzerFileInfo, ImportLookupResult } from './analyzerFileInfo'; import { AnalyzerFileInfo, ImportLookupResult } from './analyzerFileInfo';
import * as AnalyzerNodeInfo from './analyzerNodeInfo'; import * as AnalyzerNodeInfo from './analyzerNodeInfo';
import { createKeyForReference, FlowAssignment, FlowAssignmentAlias, FlowCall, FlowCondition, import {
FlowFlags, FlowLabel, FlowNode, FlowPostFinally, FlowPreFinallyGate, FlowWildcardImport, createKeyForReference,
getUniqueFlowNodeId, isCodeFlowSupportedForReference } from './codeFlow'; FlowAssignment,
import { AliasDeclaration, ClassDeclaration, DeclarationType, FunctionDeclaration, FlowAssignmentAlias,
IntrinsicType, ModuleLoaderActions, ParameterDeclaration, VariableDeclaration } from './declaration'; FlowCall,
FlowCondition,
FlowFlags,
FlowLabel,
FlowNode,
FlowPostFinally,
FlowPreFinallyGate,
FlowWildcardImport,
getUniqueFlowNodeId,
isCodeFlowSupportedForReference
} from './codeFlow';
import {
AliasDeclaration,
ClassDeclaration,
DeclarationType,
FunctionDeclaration,
IntrinsicType,
ModuleLoaderActions,
ParameterDeclaration,
VariableDeclaration
} from './declaration';
import { ImplicitImport, ImportResult, ImportType } from './importResult'; import { ImplicitImport, ImportResult, ImportType } from './importResult';
import * as ParseTreeUtils from './parseTreeUtils'; import * as ParseTreeUtils from './parseTreeUtils';
import { ParseTreeWalker } from './parseTreeWalker'; import { ParseTreeWalker } from './parseTreeWalker';
@ -147,9 +203,10 @@ export class Binder extends ParseTreeWalker {
// binding the builtins module itself. // binding the builtins module itself.
const isBuiltInModule = this._fileInfo.builtinsScope === undefined; const isBuiltInModule = this._fileInfo.builtinsScope === undefined;
this._createNewScope(isBuiltInModule ? ScopeType.Builtin : ScopeType.Module, this._createNewScope(
this._fileInfo.builtinsScope, () => { isBuiltInModule ? ScopeType.Builtin : ScopeType.Module,
this._fileInfo.builtinsScope,
() => {
AnalyzerNodeInfo.setScope(node, this._currentScope); AnalyzerNodeInfo.setScope(node, this._currentScope);
// Bind implicit names. // Bind implicit names.
@ -169,7 +226,8 @@ export class Binder extends ParseTreeWalker {
this._walkStatementsAndReportUnreachable(node.statements); this._walkStatementsAndReportUnreachable(node.statements);
AnalyzerNodeInfo.setCodeFlowExpressions(node, this._currentExecutionScopeReferenceMap); AnalyzerNodeInfo.setCodeFlowExpressions(node, this._currentExecutionScopeReferenceMap);
}); }
);
// Perform all analysis that was deferred during the first pass. // Perform all analysis that was deferred during the first pass.
this._bindDeferred(); this._bindDeferred();
@ -197,15 +255,20 @@ export class Binder extends ParseTreeWalker {
if (importResult) { if (importResult) {
if (!importResult.isImportFound) { if (!importResult.isImportFound) {
this._addDiagnostic(this._fileInfo.diagnosticSettings.reportMissingImports, this._addDiagnostic(
this._fileInfo.diagnosticSettings.reportMissingImports,
DiagnosticRule.reportMissingImports, DiagnosticRule.reportMissingImports,
`Import '${ importResult.importName }' could not be resolved`, node); `Import '${importResult.importName}' could not be resolved`,
node
);
} else if (importResult.importType === ImportType.ThirdParty) { } else if (importResult.importType === ImportType.ThirdParty) {
if (!importResult.isStubFile) { if (!importResult.isStubFile) {
const diagnostic = this._addDiagnostic( const diagnostic = this._addDiagnostic(
this._fileInfo.diagnosticSettings.reportMissingTypeStubs, this._fileInfo.diagnosticSettings.reportMissingTypeStubs,
DiagnosticRule.reportMissingTypeStubs, DiagnosticRule.reportMissingTypeStubs,
`Stub file not found for '${ importResult.importName }'`, node); `Stub file not found for '${importResult.importName}'`,
node
);
if (diagnostic) { if (diagnostic) {
// Add a diagnostic action for resolving this diagnostic. // Add a diagnostic action for resolving this diagnostic.
const createTypeStubAction: CreateTypeStubFileAction = { const createTypeStubAction: CreateTypeStubFileAction = {
@ -228,8 +291,7 @@ export class Binder extends ParseTreeWalker {
type: DeclarationType.Class, type: DeclarationType.Class,
node, node,
path: this._fileInfo.filePath, path: this._fileInfo.filePath,
range: convertOffsetsToRange(node.name.start, range: convertOffsetsToRange(node.name.start, TextRange.getEnd(node.name), this._fileInfo.lines)
TextRange.getEnd(node.name), this._fileInfo.lines)
}; };
const symbol = this._bindNameToScope(this._currentScope, node.name.value); const symbol = this._bindNameToScope(this._currentScope, node.name.value);
@ -277,8 +339,7 @@ export class Binder extends ParseTreeWalker {
isMethod: !!containingClassNode, isMethod: !!containingClassNode,
isGenerator: false, isGenerator: false,
path: this._fileInfo.filePath, path: this._fileInfo.filePath,
range: convertOffsetsToRange(node.name.start, TextRange.getEnd(node.name), range: convertOffsetsToRange(node.name.start, TextRange.getEnd(node.name), this._fileInfo.lines)
this._fileInfo.lines)
}; };
if (symbol) { if (symbol) {
@ -308,8 +369,10 @@ export class Binder extends ParseTreeWalker {
// the scope of the containing function or module when they execute. // the scope of the containing function or module when they execute.
let functionOrModuleNode: ParseNode | undefined = node.parent; let functionOrModuleNode: ParseNode | undefined = node.parent;
while (functionOrModuleNode) { while (functionOrModuleNode) {
if (functionOrModuleNode.nodeType === ParseNodeType.Module || if (
functionOrModuleNode.nodeType === ParseNodeType.Function) { functionOrModuleNode.nodeType === ParseNodeType.Module ||
functionOrModuleNode.nodeType === ParseNodeType.Function
) {
break; break;
} }
@ -358,8 +421,11 @@ export class Binder extends ParseTreeWalker {
type: DeclarationType.Parameter, type: DeclarationType.Parameter,
node: paramNode, node: paramNode,
path: this._fileInfo.filePath, path: this._fileInfo.filePath,
range: convertOffsetsToRange(paramNode.start, TextRange.getEnd(paramNode), range: convertOffsetsToRange(
this._fileInfo.lines) paramNode.start,
TextRange.getEnd(paramNode),
this._fileInfo.lines
)
}; };
symbol.addDeclaration(paramDeclaration); symbol.addDeclaration(paramDeclaration);
@ -421,8 +487,11 @@ export class Binder extends ParseTreeWalker {
type: DeclarationType.Parameter, type: DeclarationType.Parameter,
node: paramNode, node: paramNode,
path: this._fileInfo.filePath, path: this._fileInfo.filePath,
range: convertOffsetsToRange(paramNode.start, TextRange.getEnd(paramNode), range: convertOffsetsToRange(
this._fileInfo.lines) paramNode.start,
TextRange.getEnd(paramNode),
this._fileInfo.lines
)
}; };
symbol.addDeclaration(paramDeclaration); symbol.addDeclaration(paramDeclaration);
@ -478,9 +547,7 @@ export class Binder extends ParseTreeWalker {
const evaluationNode = ParseTreeUtils.getEvaluationNodeForAssignmentExpression(node); const evaluationNode = ParseTreeUtils.getEvaluationNodeForAssignmentExpression(node);
if (!evaluationNode) { if (!evaluationNode) {
this._addError( this._addError('Assignment expression must be within module, function or lambda', node);
'Assignment expression must be within module, function or lambda',
node);
} else { } else {
// Bind the name to the containing scope. This special logic is required // Bind the name to the containing scope. This special logic is required
// because of the behavior defined in PEP 572. Targets of assignment // because of the behavior defined in PEP 572. Targets of assignment
@ -498,7 +565,8 @@ export class Binder extends ParseTreeWalker {
this._addError( this._addError(
`Assignment expression target '${node.name.value}' ` + `Assignment expression target '${node.name.value}' ` +
`cannot use same name as comprehension for target`, `cannot use same name as comprehension for target`,
node.name); node.name
);
break; break;
} }
@ -649,21 +717,23 @@ export class Binder extends ParseTreeWalker {
// Determine if the test condition is always true or always false. If so, // Determine if the test condition is always true or always false. If so,
// we can treat either the then or the else clause as unconditional. // we can treat either the then or the else clause as unconditional.
const constExprValue = StaticExpressions.evaluateStaticBoolLikeExpression( const constExprValue = StaticExpressions.evaluateStaticBoolLikeExpression(
node.testExpression, this._fileInfo.executionEnvironment); node.testExpression,
this._fileInfo.executionEnvironment
);
this._bindConditional(node.testExpression, thenLabel, elseLabel); this._bindConditional(node.testExpression, thenLabel, elseLabel);
// Handle the if clause. // Handle the if clause.
this._currentFlowNode = constExprValue === false ? this._currentFlowNode =
Binder._unreachableFlowNode : this._finishFlowLabel(thenLabel); constExprValue === false ? Binder._unreachableFlowNode : this._finishFlowLabel(thenLabel);
this.walk(node.ifSuite); this.walk(node.ifSuite);
this._addAntecedent(postIfLabel, this._currentFlowNode); this._addAntecedent(postIfLabel, this._currentFlowNode);
// Now handle the else clause if it's present. If there // Now handle the else clause if it's present. If there
// are chained "else if" statements, they'll be handled // are chained "else if" statements, they'll be handled
// recursively here. // recursively here.
this._currentFlowNode = constExprValue === true ? this._currentFlowNode =
Binder._unreachableFlowNode : this._finishFlowLabel(elseLabel); constExprValue === true ? Binder._unreachableFlowNode : this._finishFlowLabel(elseLabel);
if (node.elseSuite) { if (node.elseSuite) {
this.walk(node.elseSuite); this.walk(node.elseSuite);
} }
@ -681,7 +751,9 @@ export class Binder extends ParseTreeWalker {
// Determine if the test condition is always true or always false. If so, // Determine if the test condition is always true or always false. If so,
// we can treat either the while or the else clause as unconditional. // we can treat either the while or the else clause as unconditional.
const constExprValue = StaticExpressions.evaluateStaticBoolLikeExpression( const constExprValue = StaticExpressions.evaluateStaticBoolLikeExpression(
node.testExpression, this._fileInfo.executionEnvironment); node.testExpression,
this._fileInfo.executionEnvironment
);
const preLoopLabel = this._createLoopLabel(); const preLoopLabel = this._createLoopLabel();
this._addAntecedent(preLoopLabel, this._currentFlowNode); this._addAntecedent(preLoopLabel, this._currentFlowNode);
@ -690,15 +762,15 @@ export class Binder extends ParseTreeWalker {
this._bindConditional(node.testExpression, thenLabel, elseLabel); this._bindConditional(node.testExpression, thenLabel, elseLabel);
// Handle the while clause. // Handle the while clause.
this._currentFlowNode = constExprValue === false ? this._currentFlowNode =
Binder._unreachableFlowNode : this._finishFlowLabel(thenLabel); constExprValue === false ? Binder._unreachableFlowNode : this._finishFlowLabel(thenLabel);
this._bindLoopStatement(preLoopLabel, postWhileLabel, () => { this._bindLoopStatement(preLoopLabel, postWhileLabel, () => {
this.walk(node.whileSuite); this.walk(node.whileSuite);
}); });
this._addAntecedent(preLoopLabel, this._currentFlowNode); this._addAntecedent(preLoopLabel, this._currentFlowNode);
this._currentFlowNode = constExprValue === true ? this._currentFlowNode =
Binder._unreachableFlowNode : this._finishFlowLabel(elseLabel); constExprValue === true ? Binder._unreachableFlowNode : this._finishFlowLabel(elseLabel);
if (node.elseSuite) { if (node.elseSuite) {
this.walk(node.elseSuite); this.walk(node.elseSuite);
} }
@ -737,8 +809,7 @@ export class Binder extends ParseTreeWalker {
node: node.name, node: node.name,
isConstant: isConstantName(node.name.value), isConstant: isConstantName(node.name.value),
path: this._fileInfo.filePath, path: this._fileInfo.filePath,
range: convertOffsetsToRange(node.name.start, TextRange.getEnd(node.name), range: convertOffsetsToRange(node.name.start, TextRange.getEnd(node.name), this._fileInfo.lines)
this._fileInfo.lines)
}; };
symbol.addDeclaration(declaration); symbol.addDeclaration(declaration);
} }
@ -757,9 +828,7 @@ export class Binder extends ParseTreeWalker {
visitRaise(node: RaiseNode): boolean { visitRaise(node: RaiseNode): boolean {
if (!node.typeExpression && this._nestedExceptDepth === 0) { if (!node.typeExpression && this._nestedExceptDepth === 0) {
this._addError( this._addError(`Raise requires parameter(s) when used outside of except clause `, node);
`Raise requires parameter(s) when used outside of except clause `,
node);
} }
if (node.typeExpression) { if (node.typeExpression) {
@ -893,8 +962,7 @@ export class Binder extends ParseTreeWalker {
antecedent: this._currentFlowNode, antecedent: this._currentFlowNode,
preFinallyGate preFinallyGate
}; };
this._currentFlowNode = isAfterElseAndExceptsReachable ? this._currentFlowNode = isAfterElseAndExceptsReachable ? postFinallyNode : Binder._unreachableFlowNode;
postFinallyNode : Binder._unreachableFlowNode;
} }
return false; return false;
@ -916,33 +984,34 @@ export class Binder extends ParseTreeWalker {
const unescapedResult = StringTokenUtils.getUnescapedString(stringNode.token); const unescapedResult = StringTokenUtils.getUnescapedString(stringNode.token);
unescapedResult.unescapeErrors.forEach((error: StringTokenUtils.UnescapeError) => { unescapedResult.unescapeErrors.forEach((error: StringTokenUtils.UnescapeError) => {
const start = stringNode.token.start + stringNode.token.prefixLength + const start =
stringNode.token.quoteMarkLength + error.offset; stringNode.token.start +
stringNode.token.prefixLength +
stringNode.token.quoteMarkLength +
error.offset;
const textRange = { start, length: error.length }; const textRange = { start, length: error.length };
if (error.errorType === StringTokenUtils.UnescapeErrorType.InvalidEscapeSequence) { if (error.errorType === StringTokenUtils.UnescapeErrorType.InvalidEscapeSequence) {
this._addDiagnostic( this._addDiagnostic(
this._fileInfo.diagnosticSettings.reportInvalidStringEscapeSequence, this._fileInfo.diagnosticSettings.reportInvalidStringEscapeSequence,
DiagnosticRule.reportInvalidStringEscapeSequence, DiagnosticRule.reportInvalidStringEscapeSequence,
'Unsupported escape sequence in string literal', textRange); 'Unsupported escape sequence in string literal',
} else if (error.errorType === textRange
StringTokenUtils.UnescapeErrorType.EscapeWithinFormatExpression) { );
} else if (error.errorType === StringTokenUtils.UnescapeErrorType.EscapeWithinFormatExpression) {
this._addError( this._addError(
'Escape sequence (backslash) not allowed in expression portion of f-string', 'Escape sequence (backslash) not allowed in expression portion of f-string',
textRange); textRange
} else if (error.errorType === );
StringTokenUtils.UnescapeErrorType.SingleCloseBraceWithinFormatLiteral) { } else if (
error.errorType === StringTokenUtils.UnescapeErrorType.SingleCloseBraceWithinFormatLiteral
) {
this._addError( this._addError(
'Single close brace not allowed within f-string literal; use double close brace', 'Single close brace not allowed within f-string literal; use double close brace',
textRange); textRange
} else if (error.errorType === );
StringTokenUtils.UnescapeErrorType.UnterminatedFormatExpression) { } else if (error.errorType === StringTokenUtils.UnescapeErrorType.UnterminatedFormatExpression) {
this._addError('Unterminated expression in f-string; missing close brace', textRange);
this._addError(
'Unterminated expression in f-string; missing close brace',
textRange);
} }
}); });
} }
@ -1043,11 +1112,11 @@ export class Binder extends ParseTreeWalker {
// we'll build a single declaration that describes the combined actions // we'll build a single declaration that describes the combined actions
// of both import statements, thus reflecting the behavior of the // of both import statements, thus reflecting the behavior of the
// python module loader. // python module loader.
const existingDecl = symbol.getDeclarations().find( const existingDecl = symbol
decl => decl.type === DeclarationType.Alias && .getDeclarations()
decl.firstNamePart === firstNamePartValue); .find(decl => decl.type === DeclarationType.Alias && decl.firstNamePart === firstNamePartValue);
const newDecl: AliasDeclaration = existingDecl as AliasDeclaration || { const newDecl: AliasDeclaration = (existingDecl as AliasDeclaration) || {
type: DeclarationType.Alias, type: DeclarationType.Alias,
node, node,
path: '', path: '',
@ -1072,9 +1141,9 @@ export class Binder extends ParseTreeWalker {
const namePartValue = node.module.nameParts[i].value; const namePartValue = node.module.nameParts[i].value;
// Is there an existing loader action for this name? // Is there an existing loader action for this name?
let loaderActions = curLoaderActions.implicitImports ? let loaderActions = curLoaderActions.implicitImports
curLoaderActions.implicitImports.get(namePartValue) : ? curLoaderActions.implicitImports.get(namePartValue)
undefined; : undefined;
if (!loaderActions) { if (!loaderActions) {
// Allocate a new loader action. // Allocate a new loader action.
loaderActions = { loaderActions = {
@ -1367,8 +1436,7 @@ export class Binder extends ParseTreeWalker {
// if it's the same name as a symbol in an outer scope. If so, we'll // if it's the same name as a symbol in an outer scope. If so, we'll
// create an alias node in the control flow graph. // create an alias node in the control flow graph.
for (const addedSymbol of addedSymbols) { for (const addedSymbol of addedSymbols) {
const aliasSymbol = this._currentScope.parent!. const aliasSymbol = this._currentScope.parent!.lookUpSymbol(addedSymbol[0]);
lookUpSymbol(addedSymbol[0]);
if (aliasSymbol) { if (aliasSymbol) {
this._createAssignmentAliasFlowNode(addedSymbol[1].id, aliasSymbol.id); this._createAssignmentAliasFlowNode(addedSymbol[1].id, aliasSymbol.id);
} }
@ -1411,10 +1479,11 @@ export class Binder extends ParseTreeWalker {
const expr = firstDecl.node.parent.rightExpression; const expr = firstDecl.node.parent.rightExpression;
if (expr.nodeType === ParseNodeType.List) { if (expr.nodeType === ParseNodeType.List) {
expr.entries.forEach(listEntryNode => { expr.entries.forEach(listEntryNode => {
if (listEntryNode.nodeType === ParseNodeType.StringList && if (
listEntryNode.nodeType === ParseNodeType.StringList &&
listEntryNode.strings.length === 1 && listEntryNode.strings.length === 1 &&
listEntryNode.strings[0].nodeType === ParseNodeType.String) { listEntryNode.strings[0].nodeType === ParseNodeType.String
) {
const entryName = listEntryNode.strings[0].value; const entryName = listEntryNode.strings[0].value;
if (lookupInfo.symbolTable.get(entryName)) { if (lookupInfo.symbolTable.get(entryName)) {
namesToImport.push(entryName); namesToImport.push(entryName);
@ -1527,26 +1596,29 @@ export class Binder extends ParseTreeWalker {
this._currentFalseTarget = savedFalseTarget; this._currentFalseTarget = savedFalseTarget;
if (!this._isLogicalExpression(node)) { if (!this._isLogicalExpression(node)) {
this._addAntecedent(trueTarget, this._addAntecedent(
this._createFlowConditional(FlowFlags.TrueCondition, trueTarget,
this._currentFlowNode, node)); this._createFlowConditional(FlowFlags.TrueCondition, this._currentFlowNode, node)
this._addAntecedent(falseTarget, );
this._createFlowConditional(FlowFlags.FalseCondition, this._addAntecedent(
this._currentFlowNode, node)); falseTarget,
this._createFlowConditional(FlowFlags.FalseCondition, this._currentFlowNode, node)
);
} }
} }
private _createFlowConditional(flags: FlowFlags, antecedent: FlowNode, private _createFlowConditional(flags: FlowFlags, antecedent: FlowNode, expression: ExpressionNode): FlowNode {
expression: ExpressionNode): FlowNode {
if (antecedent.flags & FlowFlags.Unreachable) { if (antecedent.flags & FlowFlags.Unreachable) {
return antecedent; return antecedent;
} }
const staticValue = StaticExpressions.evaluateStaticBoolLikeExpression( const staticValue = StaticExpressions.evaluateStaticBoolLikeExpression(
expression, this._fileInfo.executionEnvironment); expression,
if (staticValue === true && (flags & FlowFlags.FalseCondition) || this._fileInfo.executionEnvironment
staticValue === false && (flags & FlowFlags.TrueCondition)) { );
if (
(staticValue === true && flags & FlowFlags.FalseCondition) ||
(staticValue === false && flags & FlowFlags.TrueCondition)
) {
return Binder._unreachableFlowNode; return Binder._unreachableFlowNode;
} }
@ -1580,17 +1652,14 @@ export class Binder extends ParseTreeWalker {
} }
case ParseNodeType.BinaryOperation: { case ParseNodeType.BinaryOperation: {
return expression.operator === OperatorType.And || return expression.operator === OperatorType.And || expression.operator === OperatorType.Or;
expression.operator === OperatorType.Or;
} }
} }
return false; return false;
} }
private _isNarrowingExpression(expression: ExpressionNode, private _isNarrowingExpression(expression: ExpressionNode, expressionList: NarrowingExpressionNode[]): boolean {
expressionList: NarrowingExpressionNode[]): boolean {
switch (expression.nodeType) { switch (expression.nodeType) {
case ParseNodeType.Name: case ParseNodeType.Name:
case ParseNodeType.MemberAccess: { case ParseNodeType.MemberAccess: {
@ -1612,30 +1681,34 @@ export class Binder extends ParseTreeWalker {
} }
case ParseNodeType.BinaryOperation: { case ParseNodeType.BinaryOperation: {
const isOrIsNotOperator = expression.operator === OperatorType.Is || const isOrIsNotOperator =
expression.operator === OperatorType.IsNot; expression.operator === OperatorType.Is || expression.operator === OperatorType.IsNot;
const equalsOrNotEqualsOperator = expression.operator === OperatorType.Equals || const equalsOrNotEqualsOperator =
expression.operator === OperatorType.NotEquals; expression.operator === OperatorType.Equals || expression.operator === OperatorType.NotEquals;
if (isOrIsNotOperator || equalsOrNotEqualsOperator) { if (isOrIsNotOperator || equalsOrNotEqualsOperator) {
// Look for "X is None", "X is not None", "X == None", "X != None". // Look for "X is None", "X is not None", "X == None", "X != None".
// These are commonly-used patterns used in control flow. // These are commonly-used patterns used in control flow.
if (expression.rightExpression.nodeType === ParseNodeType.Constant && if (
expression.rightExpression.constType === KeywordType.None) { expression.rightExpression.nodeType === ParseNodeType.Constant &&
expression.rightExpression.constType === KeywordType.None
) {
return this._isNarrowingExpression(expression.leftExpression, expressionList); return this._isNarrowingExpression(expression.leftExpression, expressionList);
} }
// Look for "type(X) is Y" or "type(X) is not Y". // Look for "type(X) is Y" or "type(X) is not Y".
if (isOrIsNotOperator && if (
isOrIsNotOperator &&
expression.leftExpression.nodeType === ParseNodeType.Call && expression.leftExpression.nodeType === ParseNodeType.Call &&
expression.leftExpression.leftExpression.nodeType === ParseNodeType.Name && expression.leftExpression.leftExpression.nodeType === ParseNodeType.Name &&
expression.leftExpression.leftExpression.value === 'type' && expression.leftExpression.leftExpression.value === 'type' &&
expression.leftExpression.arguments.length === 1 && expression.leftExpression.arguments.length === 1 &&
expression.leftExpression.arguments[0].argumentCategory === ArgumentCategory.Simple) { expression.leftExpression.arguments[0].argumentCategory === ArgumentCategory.Simple
) {
return this._isNarrowingExpression( return this._isNarrowingExpression(
expression.leftExpression.arguments[0].valueExpression, expressionList); expression.leftExpression.arguments[0].valueExpression,
expressionList
);
} }
} }
@ -1643,8 +1716,10 @@ export class Binder extends ParseTreeWalker {
} }
case ParseNodeType.UnaryOperation: { case ParseNodeType.UnaryOperation: {
return expression.operator === OperatorType.Not && return (
this._isNarrowingExpression(expression.expression, expressionList); expression.operator === OperatorType.Not &&
this._isNarrowingExpression(expression.expression, expressionList)
);
} }
case ParseNodeType.AugmentedAssignment: { case ParseNodeType.AugmentedAssignment: {
@ -1652,18 +1727,20 @@ export class Binder extends ParseTreeWalker {
} }
case ParseNodeType.Call: { case ParseNodeType.Call: {
if (expression.leftExpression.nodeType === ParseNodeType.Name && if (
expression.leftExpression.nodeType === ParseNodeType.Name &&
(expression.leftExpression.value === 'isinstance' || (expression.leftExpression.value === 'isinstance' ||
expression.leftExpression.value === 'issubclass') && expression.leftExpression.value === 'issubclass') &&
expression.arguments.length === 2) { expression.arguments.length === 2
) {
return this._isNarrowingExpression(expression.arguments[0].valueExpression, expressionList); return this._isNarrowingExpression(expression.arguments[0].valueExpression, expressionList);
} }
if (expression.leftExpression.nodeType === ParseNodeType.Name && if (
expression.leftExpression.nodeType === ParseNodeType.Name &&
expression.leftExpression.value === 'callable' && expression.leftExpression.value === 'callable' &&
expression.arguments.length === 1) { expression.arguments.length === 1
) {
return this._isNarrowingExpression(expression.arguments[0].valueExpression, expressionList); return this._isNarrowingExpression(expression.arguments[0].valueExpression, expressionList);
} }
} }
@ -1840,8 +1917,7 @@ export class Binder extends ParseTreeWalker {
// Don't overwrite an existing symbol. // Don't overwrite an existing symbol.
let symbol = scope.lookUpSymbol(name); let symbol = scope.lookUpSymbol(name);
if (!symbol) { if (!symbol) {
symbol = scope.addSymbol(name, symbol = scope.addSymbol(name, SymbolFlags.InitiallyUnbound | SymbolFlags.ClassMember);
SymbolFlags.InitiallyUnbound | SymbolFlags.ClassMember);
if (this._fileInfo.isStubFile && isPrivateOrProtectedName(name)) { if (this._fileInfo.isStubFile && isPrivateOrProtectedName(name)) {
symbol.setIsExternallyHidden(); symbol.setIsExternallyHidden();
@ -1891,9 +1967,11 @@ export class Binder extends ParseTreeWalker {
} }
} }
private _addBuiltInSymbolToCurrentScope(nameValue: string, private _addBuiltInSymbolToCurrentScope(
node: ModuleNode | ClassNode | FunctionNode, type: IntrinsicType) { nameValue: string,
node: ModuleNode | ClassNode | FunctionNode,
type: IntrinsicType
) {
const symbol = this._addSymbolToCurrentScope(nameValue, false); const symbol = this._addSymbolToCurrentScope(nameValue, false);
if (symbol) { if (symbol) {
symbol.addDeclaration({ symbol.addDeclaration({
@ -1937,15 +2015,13 @@ export class Binder extends ParseTreeWalker {
return symbol; return symbol;
} }
private _createNewScope(scopeType: ScopeType, parentScope: Scope | undefined, private _createNewScope(scopeType: ScopeType, parentScope: Scope | undefined, callback: () => void) {
callback: () => void) {
const prevScope = this._currentScope; const prevScope = this._currentScope;
this._currentScope = new Scope(scopeType, parentScope); this._currentScope = new Scope(scopeType, parentScope);
// If this scope is an execution scope, allocate a new reference map. // If this scope is an execution scope, allocate a new reference map.
const isExecutionScope = scopeType === ScopeType.Builtin || scopeType === ScopeType.Module || const isExecutionScope =
scopeType === ScopeType.Function; scopeType === ScopeType.Builtin || scopeType === ScopeType.Module || scopeType === ScopeType.Function;
const prevReferenceMap = this._currentExecutionScopeReferenceMap; const prevReferenceMap = this._currentExecutionScopeReferenceMap;
if (isExecutionScope) { if (isExecutionScope) {
@ -1988,10 +2064,8 @@ export class Binder extends ParseTreeWalker {
let symbol = memberAccessInfo.classScope.lookUpSymbol(name.value); let symbol = memberAccessInfo.classScope.lookUpSymbol(name.value);
if (!symbol) { if (!symbol) {
symbol = memberAccessInfo.classScope.addSymbol(name.value, symbol = memberAccessInfo.classScope.addSymbol(name.value, SymbolFlags.InitiallyUnbound);
SymbolFlags.InitiallyUnbound); const honorPrivateNaming = this._fileInfo.diagnosticSettings.reportPrivateUsage !== 'none';
const honorPrivateNaming =
this._fileInfo.diagnosticSettings.reportPrivateUsage !== 'none';
if (isPrivateOrProtectedName(name.value) && honorPrivateNaming) { if (isPrivateOrProtectedName(name.value) && honorPrivateNaming) {
symbol.setIsPrivateMember(); symbol.setIsPrivateMember();
} }
@ -2009,9 +2083,11 @@ export class Binder extends ParseTreeWalker {
isConstant: isConstantName(name.value), isConstant: isConstantName(name.value),
inferredTypeSource: source, inferredTypeSource: source,
path: this._fileInfo.filePath, path: this._fileInfo.filePath,
range: convertOffsetsToRange(target.memberName.start, range: convertOffsetsToRange(
target.memberName.start,
target.memberName.start + target.memberName.length, target.memberName.start + target.memberName.length,
this._fileInfo.lines) this._fileInfo.lines
)
}; };
symbol.addDeclaration(declaration); symbol.addDeclaration(declaration);
} }
@ -2059,8 +2135,7 @@ export class Binder extends ParseTreeWalker {
isConstant: isConstantName(name.value), isConstant: isConstantName(name.value),
isFinal: finalInfo.isFinal, isFinal: finalInfo.isFinal,
path: this._fileInfo.filePath, path: this._fileInfo.filePath,
typeAnnotationNode: finalInfo.isFinal ? typeAnnotationNode: finalInfo.isFinal ? finalInfo.finalTypeNode : typeAnnotation,
finalInfo.finalTypeNode : typeAnnotation,
range: convertOffsetsToRange(name.start, TextRange.getEnd(name), this._fileInfo.lines) range: convertOffsetsToRange(name.start, TextRange.getEnd(name), this._fileInfo.lines)
}; };
symbolWithScope.symbol.addDeclaration(declaration); symbolWithScope.symbol.addDeclaration(declaration);
@ -2085,10 +2160,8 @@ export class Binder extends ParseTreeWalker {
let symbol = memberAccessInfo.classScope.lookUpSymbol(name.value); let symbol = memberAccessInfo.classScope.lookUpSymbol(name.value);
if (!symbol) { if (!symbol) {
symbol = memberAccessInfo.classScope.addSymbol(name.value, symbol = memberAccessInfo.classScope.addSymbol(name.value, SymbolFlags.InitiallyUnbound);
SymbolFlags.InitiallyUnbound); const honorPrivateNaming = this._fileInfo.diagnosticSettings.reportPrivateUsage !== 'none';
const honorPrivateNaming =
this._fileInfo.diagnosticSettings.reportPrivateUsage !== 'none';
if (isPrivateOrProtectedName(name.value) && honorPrivateNaming) { if (isPrivateOrProtectedName(name.value) && honorPrivateNaming) {
symbol.setIsPrivateMember(); symbol.setIsPrivateMember();
} }
@ -2107,11 +2180,12 @@ export class Binder extends ParseTreeWalker {
isConstant: isConstantName(name.value), isConstant: isConstantName(name.value),
isFinal: finalInfo.isFinal, isFinal: finalInfo.isFinal,
path: this._fileInfo.filePath, path: this._fileInfo.filePath,
typeAnnotationNode: finalInfo.isFinal ? typeAnnotationNode: finalInfo.isFinal ? finalInfo.finalTypeNode : typeAnnotation,
finalInfo.finalTypeNode : typeAnnotation, range: convertOffsetsToRange(
range: convertOffsetsToRange(target.memberName.start, target.memberName.start,
target.memberName.start + target.memberName.length, target.memberName.start + target.memberName.length,
this._fileInfo.lines) this._fileInfo.lines
)
}; };
symbol.addDeclaration(declaration); symbol.addDeclaration(declaration);
@ -2122,9 +2196,7 @@ export class Binder extends ParseTreeWalker {
} }
if (!declarationHandled) { if (!declarationHandled) {
this._addError( this._addError(`Type annotation not supported for this type of expression`, typeAnnotation);
`Type annotation not supported for this type of expression`,
typeAnnotation);
} }
} }
@ -2145,15 +2217,14 @@ export class Binder extends ParseTreeWalker {
isFinal = true; isFinal = true;
} }
} else if (typeAnnotation.nodeType === ParseNodeType.MemberAccess) { } else if (typeAnnotation.nodeType === ParseNodeType.MemberAccess) {
if (typeAnnotation.leftExpression.nodeType === ParseNodeType.Name && if (
typeAnnotation.leftExpression.nodeType === ParseNodeType.Name &&
typeAnnotation.leftExpression.value === 'typing' && typeAnnotation.leftExpression.value === 'typing' &&
typeAnnotation.memberName.value === 'Final') { typeAnnotation.memberName.value === 'Final'
) {
isFinal = true; isFinal = true;
} }
} else if (typeAnnotation.nodeType === ParseNodeType.Index && } else if (typeAnnotation.nodeType === ParseNodeType.Index && typeAnnotation.items.items.length === 1) {
typeAnnotation.items.items.length === 1) {
// Recursively call to see if the base expression is "Final". // Recursively call to see if the base expression is "Final".
const finalInfo = this._isAnnotationFinal(typeAnnotation.baseExpression); const finalInfo = this._isAnnotationFinal(typeAnnotation.baseExpression);
if (finalInfo.isFinal) { if (finalInfo.isFinal) {
@ -2249,9 +2320,9 @@ export class Binder extends ParseTreeWalker {
private _addImplicitImportsToLoaderActions(importResult: ImportResult, loaderActions: ModuleLoaderActions) { private _addImplicitImportsToLoaderActions(importResult: ImportResult, loaderActions: ModuleLoaderActions) {
importResult.implicitImports.forEach(implicitImport => { importResult.implicitImports.forEach(implicitImport => {
const existingLoaderAction = loaderActions.implicitImports ? const existingLoaderAction = loaderActions.implicitImports
loaderActions.implicitImports.get(implicitImport.name) : ? loaderActions.implicitImports.get(implicitImport.name)
undefined; : undefined;
if (existingLoaderAction) { if (existingLoaderAction) {
existingLoaderAction.path = implicitImport.path; existingLoaderAction.path = implicitImport.path;
} else { } else {
@ -2291,15 +2362,15 @@ export class Binder extends ParseTreeWalker {
const assignedNameNode = annotationNode.valueExpression; const assignedNameNode = annotationNode.valueExpression;
const specialTypes: { [name: string]: boolean } = { const specialTypes: { [name: string]: boolean } = {
'Tuple': true, Tuple: true,
'Generic': true, Generic: true,
'Protocol': true, Protocol: true,
'Callable': true, Callable: true,
'Type': true, Type: true,
'ClassVar': true, ClassVar: true,
'Final': true, Final: true,
'Literal': true, Literal: true,
'TypedDict': true TypedDict: true
}; };
const assignedName = assignedNameNode.value; const assignedName = assignedNameNode.value;
@ -2314,8 +2385,11 @@ export class Binder extends ParseTreeWalker {
type: DeclarationType.SpecialBuiltInClass, type: DeclarationType.SpecialBuiltInClass,
node: annotationNode, node: annotationNode,
path: this._fileInfo.filePath, path: this._fileInfo.filePath,
range: convertOffsetsToRange(annotationNode.start, range: convertOffsetsToRange(
TextRange.getEnd(annotationNode), this._fileInfo.lines) annotationNode.start,
TextRange.getEnd(annotationNode),
this._fileInfo.lines
)
}); });
} }
return true; return true;
@ -2348,13 +2422,11 @@ export class Binder extends ParseTreeWalker {
const functionNode = ParseTreeUtils.getEnclosingFunction(node); const functionNode = ParseTreeUtils.getEnclosingFunction(node);
if (!functionNode) { if (!functionNode) {
this._addError( this._addError(`'yield' not allowed outside of a function`, node);
`'yield' not allowed outside of a function`, node);
} else if (functionNode.isAsync && node.nodeType === ParseNodeType.YieldFrom) { } else if (functionNode.isAsync && node.nodeType === ParseNodeType.YieldFrom) {
// PEP 525 indicates that 'yield from' is not allowed in an // PEP 525 indicates that 'yield from' is not allowed in an
// async function. // async function.
this._addError( this._addError(`'yield from' not allowed in an async function`, node);
`'yield from' not allowed in an async function`, node);
} }
if (this._targetFunctionDeclaration) { if (this._targetFunctionDeclaration) {
@ -2372,9 +2444,7 @@ export class Binder extends ParseTreeWalker {
AnalyzerNodeInfo.setFlowNode(node, this._currentFlowNode); AnalyzerNodeInfo.setFlowNode(node, this._currentFlowNode);
} }
private _addDiagnostic(diagLevel: DiagnosticLevel, rule: string, private _addDiagnostic(diagLevel: DiagnosticLevel, rule: string, message: string, textRange: TextRange) {
message: string, textRange: TextRange) {
if (diagLevel === 'error') { if (diagLevel === 'error') {
const diagnostic = this._addError(message, textRange); const diagnostic = this._addError(message, textRange);
diagnostic.setRule(rule); diagnostic.setRule(rule);
@ -2388,8 +2458,7 @@ export class Binder extends ParseTreeWalker {
} }
private _addUnusedCode(textRange: TextRange) { private _addUnusedCode(textRange: TextRange) {
return this._fileInfo.diagnosticSink.addUnusedCodeWithTextRange( return this._fileInfo.diagnosticSink.addUnusedCodeWithTextRange('Code is unreachable', textRange);
'Code is unreachable', textRange);
} }
private _addError(message: string, textRange: TextRange) { private _addError(message: string, textRange: TextRange) {

413
server/src/analyzer/checker.ts
View File

@ -17,13 +17,48 @@ import { assert } from '../common/debug';
import { Diagnostic, DiagnosticAddendum } from '../common/diagnostic'; import { Diagnostic, DiagnosticAddendum } from '../common/diagnostic';
import { DiagnosticRule } from '../common/diagnosticRules'; import { DiagnosticRule } from '../common/diagnosticRules';
import { TextRange } from '../common/textRange'; import { TextRange } from '../common/textRange';
import { AssertNode, AssignmentExpressionNode, AssignmentNode, AugmentedAssignmentNode, import {
BinaryOperationNode, CallNode, ClassNode, DelNode, ErrorNode, ExceptNode, AssertNode,
FormatStringNode, ForNode, FunctionNode, IfNode, ImportAsNode, ImportFromAsNode, ImportFromNode, AssignmentExpressionNode,
IndexNode, LambdaNode, ListComprehensionNode, MemberAccessNode, ModuleNode, NameNode, AssignmentNode,
ParameterCategory, ParseNode, ParseNodeType, RaiseNode, ReturnNode, SliceNode, StringListNode, AugmentedAssignmentNode,
SuiteNode, TernaryNode, TupleNode, TypeAnnotationNode, UnaryOperationNode, UnpackNode, BinaryOperationNode,
WhileNode, WithNode, YieldFromNode, YieldNode } from '../parser/parseNodes'; CallNode,
ClassNode,
DelNode,
ErrorNode,
ExceptNode,
FormatStringNode,
ForNode,
FunctionNode,
IfNode,
ImportAsNode,
ImportFromAsNode,
ImportFromNode,
IndexNode,
LambdaNode,
ListComprehensionNode,
MemberAccessNode,
ModuleNode,
NameNode,
ParameterCategory,
ParseNode,
ParseNodeType,
RaiseNode,
ReturnNode,
SliceNode,
StringListNode,
SuiteNode,
TernaryNode,
TupleNode,
TypeAnnotationNode,
UnaryOperationNode,
UnpackNode,
WhileNode,
WithNode,
YieldFromNode,
YieldNode
} from '../parser/parseNodes';
import { AnalyzerFileInfo } from './analyzerFileInfo'; import { AnalyzerFileInfo } from './analyzerFileInfo';
import * as AnalyzerNodeInfo from './analyzerNodeInfo'; import * as AnalyzerNodeInfo from './analyzerNodeInfo';
import { Declaration, DeclarationType } from './declaration'; import { Declaration, DeclarationType } from './declaration';
@ -36,11 +71,31 @@ import { Symbol } from './symbol';
import * as SymbolNameUtils from './symbolNameUtils'; import * as SymbolNameUtils from './symbolNameUtils';
import { getLastTypedDeclaredForSymbol, isFinalVariable } from './symbolUtils'; import { getLastTypedDeclaredForSymbol, isFinalVariable } from './symbolUtils';
import { TypeEvaluator } from './typeEvaluator'; import { TypeEvaluator } from './typeEvaluator';
import { ClassType, combineTypes, FunctionType, isAnyOrUnknown, isNoneOrNever, isTypeSame, import {
NoneType, ObjectType, Type, TypeCategory, UnknownType } from './types'; ClassType,
import { containsUnknown, derivesFromClassRecursive, doForSubtypes, combineTypes,
getDeclaredGeneratorReturnType, getDeclaredGeneratorYieldType, getSymbolFromBaseClasses, FunctionType,
isNoReturnType, isProperty, specializeType, transformTypeObjectToClass } from './typeUtils'; isAnyOrUnknown,
isNoneOrNever,
isTypeSame,
NoneType,
ObjectType,
Type,
TypeCategory,
UnknownType
} from './types';
import {
containsUnknown,
derivesFromClassRecursive,
doForSubtypes,
getDeclaredGeneratorReturnType,
getDeclaredGeneratorYieldType,
getSymbolFromBaseClasses,
isNoReturnType,
isProperty,
specializeType,
transformTypeObjectToClass
} from './typeUtils';
export class Checker extends ParseTreeWalker { export class Checker extends ParseTreeWalker {
private readonly _moduleNode: ModuleNode; private readonly _moduleNode: ModuleNode;
@ -52,7 +107,6 @@ export class Checker extends ParseTreeWalker {
private _scopedNodes: AnalyzerNodeInfo.ScopedNode[] = []; private _scopedNodes: AnalyzerNodeInfo.ScopedNode[] = [];
constructor(node: ModuleNode, evaluator: TypeEvaluator) { constructor(node: ModuleNode, evaluator: TypeEvaluator) {
super(); super();
this._moduleNode = node; this._moduleNode = node;
@ -114,7 +168,8 @@ export class Checker extends ParseTreeWalker {
this._fileInfo.diagnosticSettings.reportUnknownParameterType, this._fileInfo.diagnosticSettings.reportUnknownParameterType,
DiagnosticRule.reportUnknownParameterType, DiagnosticRule.reportUnknownParameterType,
`Type of parameter '${param.name.value}' is unknown`, `Type of parameter '${param.name.value}' is unknown`,
param.name); param.name
);
} else if (containsUnknown(paramType)) { } else if (containsUnknown(paramType)) {
const diagAddendum = new DiagnosticAddendum(); const diagAddendum = new DiagnosticAddendum();
diagAddendum.addMessage(`Parameter type is '${this._evaluator.printType(paramType)}'`); diagAddendum.addMessage(`Parameter type is '${this._evaluator.printType(paramType)}'`);
@ -122,7 +177,8 @@ export class Checker extends ParseTreeWalker {
this._fileInfo.diagnosticSettings.reportUnknownParameterType, this._fileInfo.diagnosticSettings.reportUnknownParameterType,
DiagnosticRule.reportUnknownParameterType, DiagnosticRule.reportUnknownParameterType,
`Type of parameter '${param.name.value}' is partially unknown` + diagAddendum.getString(), `Type of parameter '${param.name.value}' is partially unknown` + diagAddendum.getString(),
param.name); param.name
);
} }
} }
}); });
@ -181,14 +237,16 @@ export class Checker extends ParseTreeWalker {
this._fileInfo.diagnosticSettings.reportUnknownLambdaType, this._fileInfo.diagnosticSettings.reportUnknownLambdaType,
DiagnosticRule.reportUnknownLambdaType, DiagnosticRule.reportUnknownLambdaType,
`Type of '${param.name.value}' is unknown`, `Type of '${param.name.value}' is unknown`,
param.name); param.name
);
} else if (containsUnknown(paramType)) { } else if (containsUnknown(paramType)) {
this._evaluator.addDiagnostic( this._evaluator.addDiagnostic(
this._fileInfo.diagnosticSettings.reportUnknownLambdaType, this._fileInfo.diagnosticSettings.reportUnknownLambdaType,
DiagnosticRule.reportUnknownLambdaType, DiagnosticRule.reportUnknownLambdaType,
`Type of '${param.name.value}', ` + `Type of '${param.name.value}', ` +
`'${this._evaluator.printType(paramType)}', is partially unknown`, `'${this._evaluator.printType(paramType)}', is partially unknown`,
param.name); param.name
);
} }
} }
} }
@ -200,13 +258,16 @@ export class Checker extends ParseTreeWalker {
this._evaluator.addDiagnostic( this._evaluator.addDiagnostic(
this._fileInfo.diagnosticSettings.reportUnknownLambdaType, this._fileInfo.diagnosticSettings.reportUnknownLambdaType,
DiagnosticRule.reportUnknownLambdaType, DiagnosticRule.reportUnknownLambdaType,
`Type of lambda expression is unknown`, node.expression); `Type of lambda expression is unknown`,
node.expression
);
} else if (containsUnknown(returnType)) { } else if (containsUnknown(returnType)) {
this._evaluator.addDiagnostic( this._evaluator.addDiagnostic(
this._fileInfo.diagnosticSettings.reportUnknownLambdaType, this._fileInfo.diagnosticSettings.reportUnknownLambdaType,
DiagnosticRule.reportUnknownLambdaType, DiagnosticRule.reportUnknownLambdaType,
`Type of lambda expression, '${this._evaluator.printType(returnType)}', is partially unknown`, `Type of lambda expression, '${this._evaluator.printType(returnType)}', is partially unknown`,
node.expression); node.expression
);
} }
} }
@ -225,7 +286,8 @@ export class Checker extends ParseTreeWalker {
this._fileInfo.diagnosticSettings.reportCallInDefaultInitializer, this._fileInfo.diagnosticSettings.reportCallInDefaultInitializer,
DiagnosticRule.reportCallInDefaultInitializer, DiagnosticRule.reportCallInDefaultInitializer,
`Function calls within default value initializer are not permitted`, `Function calls within default value initializer are not permitted`,
node); node
);
} }
return true; return true;
@ -263,9 +325,9 @@ export class Checker extends ParseTreeWalker {
let returnType: Type; let returnType: Type;
const enclosingFunctionNode = ParseTreeUtils.getEnclosingFunction(node); const enclosingFunctionNode = ParseTreeUtils.getEnclosingFunction(node);
const declaredReturnType = enclosingFunctionNode ? const declaredReturnType = enclosingFunctionNode
this._evaluator.getFunctionDeclaredReturnType(enclosingFunctionNode) : ? this._evaluator.getFunctionDeclaredReturnType(enclosingFunctionNode)
undefined; : undefined;
if (node.returnExpression) { if (node.returnExpression) {
returnType = this._evaluator.getType(node.returnExpression) || UnknownType.create(); returnType = this._evaluator.getType(node.returnExpression) || UnknownType.create();
@ -279,7 +341,8 @@ export class Checker extends ParseTreeWalker {
if (isNoReturnType(declaredReturnType)) { if (isNoReturnType(declaredReturnType)) {
this._evaluator.addError( this._evaluator.addError(
`Function with declared return type 'NoReturn' cannot include a return statement`, `Function with declared return type 'NoReturn' cannot include a return statement`,
node); node
);
} else { } else {
const diagAddendum = new DiagnosticAddendum(); const diagAddendum = new DiagnosticAddendum();
@ -291,7 +354,8 @@ export class Checker extends ParseTreeWalker {
`Expression of type '${this._evaluator.printType(returnType)}' cannot be assigned ` + `Expression of type '${this._evaluator.printType(returnType)}' cannot be assigned ` +
`to return type '${this._evaluator.printType(specializedDeclaredType)}'` + `to return type '${this._evaluator.printType(specializedDeclaredType)}'` +
diagAddendum.getString(), diagAddendum.getString(),
node.returnExpression ? node.returnExpression : node); node.returnExpression ? node.returnExpression : node
);
} }
} }
} }
@ -300,13 +364,16 @@ export class Checker extends ParseTreeWalker {
this._evaluator.addDiagnostic( this._evaluator.addDiagnostic(
this._fileInfo.diagnosticSettings.reportUnknownVariableType, this._fileInfo.diagnosticSettings.reportUnknownVariableType,
DiagnosticRule.reportUnknownVariableType, DiagnosticRule.reportUnknownVariableType,
`Return type is unknown`, node.returnExpression!); `Return type is unknown`,
node.returnExpression!
);
} else if (containsUnknown(returnType)) { } else if (containsUnknown(returnType)) {
this._evaluator.addDiagnostic( this._evaluator.addDiagnostic(
this._fileInfo.diagnosticSettings.reportUnknownVariableType, this._fileInfo.diagnosticSettings.reportUnknownVariableType,
DiagnosticRule.reportUnknownVariableType, DiagnosticRule.reportUnknownVariableType,
`Return type, '${this._evaluator.printType(returnType)}', is partially unknown`, `Return type, '${this._evaluator.printType(returnType)}', is partially unknown`,
node.returnExpression!); node.returnExpression!
);
} }
} }
@ -314,8 +381,7 @@ export class Checker extends ParseTreeWalker {
} }
visitYield(node: YieldNode) { visitYield(node: YieldNode) {
const yieldType = node.expression ? const yieldType = node.expression ? this._evaluator.getType(node.expression) : NoneType.create();
this._evaluator.getType(node.expression) : NoneType.create();
// Wrap the yield type in an Iterator. // Wrap the yield type in an Iterator.
let adjYieldType = yieldType; let adjYieldType = yieldType;
@ -354,14 +420,20 @@ export class Checker extends ParseTreeWalker {
if (!isAnyOrUnknown(subtype)) { if (!isAnyOrUnknown(subtype)) {
if (subtype.category === TypeCategory.Class) { if (subtype.category === TypeCategory.Class) {
if (!derivesFromClassRecursive(subtype, baseExceptionType)) { if (!derivesFromClassRecursive(subtype, baseExceptionType)) {
diagAddendum.addMessage(`'${ this._evaluator.printType(subtype) }' does not derive from BaseException`); diagAddendum.addMessage(
`'${this._evaluator.printType(subtype)}' does not derive from BaseException`
);
} }
} else if (subtype.category === TypeCategory.Object) { } else if (subtype.category === TypeCategory.Object) {
if (!derivesFromClassRecursive(subtype.classType, baseExceptionType)) { if (!derivesFromClassRecursive(subtype.classType, baseExceptionType)) {
diagAddendum.addMessage(`'${ this._evaluator.printType(subtype) }' does not derive from BaseException`); diagAddendum.addMessage(
`'${this._evaluator.printType(subtype)}' does not derive from BaseException`
);
} }
} else { } else {
diagAddendum.addMessage(`'${ this._evaluator.printType(subtype) }' does not derive from BaseException`); diagAddendum.addMessage(
`'${this._evaluator.printType(subtype)}' does not derive from BaseException`
);
} }
} }
@ -371,7 +443,8 @@ export class Checker extends ParseTreeWalker {
if (diagAddendum.getMessageCount() > 0) { if (diagAddendum.getMessageCount() > 0) {
this._evaluator.addError( this._evaluator.addError(
`Expected exception class or object` + diagAddendum.getString(), `Expected exception class or object` + diagAddendum.getString(),
node.typeExpression); node.typeExpression
);
} }
} }
} }
@ -387,10 +460,14 @@ export class Checker extends ParseTreeWalker {
if (!isAnyOrUnknown(subtype) && !isNoneOrNever(subtype)) { if (!isAnyOrUnknown(subtype) && !isNoneOrNever(subtype)) {
if (subtype.category === TypeCategory.Object) { if (subtype.category === TypeCategory.Object) {
if (!derivesFromClassRecursive(subtype.classType, baseExceptionType)) { if (!derivesFromClassRecursive(subtype.classType, baseExceptionType)) {
diagAddendum.addMessage(`'${ this._evaluator.printType(subtype) }' does not derive from BaseException`); diagAddendum.addMessage(
`'${this._evaluator.printType(subtype)}' does not derive from BaseException`
);
} }
} else { } else {
diagAddendum.addMessage(`'${ this._evaluator.printType(subtype) }' does not derive from BaseException`); diagAddendum.addMessage(
`'${this._evaluator.printType(subtype)}' does not derive from BaseException`
);
} }
} }
@ -400,7 +477,8 @@ export class Checker extends ParseTreeWalker {
if (diagAddendum.getMessageCount() > 0) { if (diagAddendum.getMessageCount() > 0) {
this._evaluator.addError( this._evaluator.addError(
`Expected exception object or None` + diagAddendum.getString(), `Expected exception object or None` + diagAddendum.getString(),
node.valueExpression); node.valueExpression
);
} }
} }
} }
@ -430,10 +508,13 @@ export class Checker extends ParseTreeWalker {
if (type && type.category === TypeCategory.Object) { if (type && type.category === TypeCategory.Object) {
if (ClassType.isBuiltIn(type.classType, 'Tuple') && type.classType.typeArguments) { if (ClassType.isBuiltIn(type.classType, 'Tuple') && type.classType.typeArguments) {
if (type.classType.typeArguments.length > 0) { if (type.classType.typeArguments.length > 0) {
this._evaluator.addDiagnosticForTextRange(this._fileInfo, this._evaluator.addDiagnosticForTextRange(
this._fileInfo,
this._fileInfo.diagnosticSettings.reportAssertAlwaysTrue, this._fileInfo.diagnosticSettings.reportAssertAlwaysTrue,
DiagnosticRule.reportAssertAlwaysTrue, DiagnosticRule.reportAssertAlwaysTrue,
`Assert expression always evaluates to true`, node.testExpression); `Assert expression always evaluates to true`,
node.testExpression
);
} }
} }
} }
@ -501,10 +582,13 @@ export class Checker extends ParseTreeWalker {
} }
if (node.strings.length > 1) { if (node.strings.length > 1) {
this._evaluator.addDiagnosticForTextRange(this._fileInfo, this._evaluator.addDiagnosticForTextRange(
this._fileInfo,
this._fileInfo.diagnosticSettings.reportImplicitStringConcatenation, this._fileInfo.diagnosticSettings.reportImplicitStringConcatenation,
DiagnosticRule.reportImplicitStringConcatenation, DiagnosticRule.reportImplicitStringConcatenation,
`Implicit string concatenation not allowed`, node); `Implicit string concatenation not allowed`,
node
);
} }
return true; return true;
@ -591,12 +675,12 @@ export class Checker extends ParseTreeWalker {
} else if (exceptionType.category === TypeCategory.Class) { } else if (exceptionType.category === TypeCategory.Class) {
if (!derivesFromBaseException(exceptionType)) { if (!derivesFromBaseException(exceptionType)) {
diagAddendum.addMessage( diagAddendum.addMessage(
`'${this._evaluator.printType(exceptionType)}' does not derive from BaseException`); `'${this._evaluator.printType(exceptionType)}' does not derive from BaseException`
);
} }
resultingExceptionType = ObjectType.create(exceptionType); resultingExceptionType = ObjectType.create(exceptionType);
} else if (exceptionType.category === TypeCategory.Object) { } else if (exceptionType.category === TypeCategory.Object) {
const iterableType = this._evaluator.getTypeFromIterable( const iterableType = this._evaluator.getTypeFromIterable(exceptionType, false, errorNode, false);
exceptionType, false, errorNode, false);
resultingExceptionType = doForSubtypes(iterableType, subtype => { resultingExceptionType = doForSubtypes(iterableType, subtype => {
if (isAnyOrUnknown(subtype)) { if (isAnyOrUnknown(subtype)) {
@ -607,23 +691,25 @@ export class Checker extends ParseTreeWalker {
if (transformedSubtype.category === TypeCategory.Class) { if (transformedSubtype.category === TypeCategory.Class) {
if (!derivesFromBaseException(transformedSubtype)) { if (!derivesFromBaseException(transformedSubtype)) {
diagAddendum.addMessage( diagAddendum.addMessage(
`'${this._evaluator.printType(exceptionType)}' does not derive from BaseException`); `'${this._evaluator.printType(exceptionType)}' does not derive from BaseException`
);
} }
return ObjectType.create(transformedSubtype); return ObjectType.create(transformedSubtype);
} }
diagAddendum.addMessage( diagAddendum.addMessage(
`'${this._evaluator.printType(exceptionType)}' does not derive from BaseException`); `'${this._evaluator.printType(exceptionType)}' does not derive from BaseException`
);
return UnknownType.create(); return UnknownType.create();
}); });
} }
if (diagAddendum.getMessageCount() > 0) { if (diagAddendum.getMessageCount() > 0) {
this._evaluator.addError( this._evaluator.addError(
`'${this._evaluator.printType(exceptionType)}' is not valid exception class` + `'${this._evaluator.printType(exceptionType)}' is not valid exception class` + diagAddendum.getString(),
diagAddendum.getString(), errorNode
errorNode); );
} }
return resultingExceptionType || UnknownType.create(); return resultingExceptionType || UnknownType.create();
@ -660,20 +746,14 @@ export class Checker extends ParseTreeWalker {
decls.forEach(decl => { decls.forEach(decl => {
if (isFinalVariableDeclaration(decl)) { if (isFinalVariableDeclaration(decl)) {
if (sawFinal) { if (sawFinal) {
this._evaluator.addError( this._evaluator.addError(`'${name}' was previously declared as Final`, decl.node);
`'${ name }' was previously declared as Final`,
decl.node
);
} }
sawFinal = true; sawFinal = true;
} }
if (decl.type === DeclarationType.Variable && decl.inferredTypeSource) { if (decl.type === DeclarationType.Variable && decl.inferredTypeSource) {
if (sawAssignment) { if (sawAssignment) {
this._evaluator.addError( this._evaluator.addError(`'${name}' is declared Final and can be assigned only once`, decl.node);
`'${ name }' is declared Final and can be assigned only once`,
decl.node
);
} }
sawAssignment = true; sawAssignment = true;
} }
@ -683,10 +763,7 @@ export class Checker extends ParseTreeWalker {
if (!sawAssignment && !this._fileInfo.isStubFile) { if (!sawAssignment && !this._fileInfo.isStubFile) {
const firstDecl = decls.find(decl => decl.type === DeclarationType.Variable && decl.isFinal); const firstDecl = decls.find(decl => decl.type === DeclarationType.Variable && decl.isFinal);
if (firstDecl) { if (firstDecl) {
this._evaluator.addError( this._evaluator.addError(`'${name}' is declared Final, but value is not assigned`, firstDecl.node);
`'${ name }' is declared Final, but value is not assigned`,
firstDecl.node
);
} }
} }
} }
@ -729,9 +806,7 @@ export class Checker extends ParseTreeWalker {
const addPrimaryDeclInfo = (diag?: Diagnostic) => { const addPrimaryDeclInfo = (diag?: Diagnostic) => {
if (diag) { if (diag) {
let primaryDeclNode: ParseNode | undefined; let primaryDeclNode: ParseNode | undefined;
if (primaryDecl.type === DeclarationType.Function || if (primaryDecl.type === DeclarationType.Function || primaryDecl.type === DeclarationType.Class) {
primaryDecl.type === DeclarationType.Class) {
primaryDeclNode = primaryDecl.node.name; primaryDeclNode = primaryDecl.node.name;
} else if (primaryDecl.type === DeclarationType.Variable) { } else if (primaryDecl.type === DeclarationType.Variable) {
if (primaryDecl.node.nodeType === ParseNodeType.Name) { if (primaryDecl.node.nodeType === ParseNodeType.Name) {
@ -744,8 +819,7 @@ export class Checker extends ParseTreeWalker {
} }
if (primaryDeclNode) { if (primaryDeclNode) {
diag.addRelatedInfo(`See ${ primaryDeclType }declaration`, diag.addRelatedInfo(`See ${primaryDeclType}declaration`, primaryDecl.path, primaryDecl.range);
primaryDecl.path, primaryDecl.range);
} }
} }
}; };
@ -753,8 +827,7 @@ export class Checker extends ParseTreeWalker {
for (const otherDecl of otherDecls) { for (const otherDecl of otherDecls) {
if (otherDecl.type === DeclarationType.Class) { if (otherDecl.type === DeclarationType.Class) {
const diag = this._evaluator.addError( const diag = this._evaluator.addError(
`Class declaration '${ name }' is obscured by a ${ primaryDeclType }` + `Class declaration '${name}' is obscured by a ${primaryDeclType}` + `declaration of the same name`,
`declaration of the same name`,
otherDecl.node.name otherDecl.node.name
); );
addPrimaryDeclInfo(diag); addPrimaryDeclInfo(diag);
@ -768,8 +841,7 @@ export class Checker extends ParseTreeWalker {
} else if (otherDecl.type === DeclarationType.Parameter) { } else if (otherDecl.type === DeclarationType.Parameter) {
if (otherDecl.node.name) { if (otherDecl.node.name) {
const diag = this._evaluator.addError( const diag = this._evaluator.addError(
`Parameter '${ name }' is obscured by a ${ primaryDeclType }` + `Parameter '${name}' is obscured by a ${primaryDeclType}` + `declaration of the same name`,
`declaration of the same name`,
otherDecl.node.name otherDecl.node.name
); );
addPrimaryDeclInfo(diag); addPrimaryDeclInfo(diag);
@ -830,8 +902,7 @@ export class Checker extends ParseTreeWalker {
const decls = symbol.getDeclarations(); const decls = symbol.getDeclarations();
decls.forEach(decl => { decls.forEach(decl => {
this._conditionallyReportUnusedDeclaration(decl, this._conditionallyReportUnusedDeclaration(decl, this._isSymbolPrivate(name, scopeType));
this._isSymbolPrivate(name, scopeType));
}); });
} }
@ -856,12 +927,17 @@ export class Checker extends ParseTreeWalker {
const textRange: TextRange = { start: nameParts[0].start, length: nameParts[0].length }; const textRange: TextRange = { start: nameParts[0].start, length: nameParts[0].length };
TextRange.extend(textRange, nameParts[nameParts.length - 1]); TextRange.extend(textRange, nameParts[nameParts.length - 1]);
this._fileInfo.diagnosticSink.addUnusedCodeWithTextRange( this._fileInfo.diagnosticSink.addUnusedCodeWithTextRange(
`'${ multipartName }' is not accessed`, textRange); `'${multipartName}' is not accessed`,
textRange
);
this._evaluator.addDiagnosticForTextRange(this._fileInfo, this._evaluator.addDiagnosticForTextRange(
this._fileInfo,
this._fileInfo.diagnosticSettings.reportUnusedImport, this._fileInfo.diagnosticSettings.reportUnusedImport,
DiagnosticRule.reportUnusedImport, DiagnosticRule.reportUnusedImport,
`Import '${ multipartName }' is not accessed`, textRange); `Import '${multipartName}' is not accessed`,
textRange
);
return; return;
} }
} }
@ -870,10 +946,11 @@ export class Checker extends ParseTreeWalker {
// unused imports. Don't report these because they're in generated // unused imports. Don't report these because they're in generated
// files that shouldn't be edited. // files that shouldn't be edited.
const importFrom = decl.node.parent as ImportFromNode; const importFrom = decl.node.parent as ImportFromNode;
if (importFrom.module.nameParts.length === 0 || if (
importFrom.module.nameParts[0].value !== '__future__' && importFrom.module.nameParts.length === 0 ||
!this._fileInfo.filePath.endsWith('_pb2.py')) { (importFrom.module.nameParts[0].value !== '__future__' &&
!this._fileInfo.filePath.endsWith('_pb2.py'))
) {
nameNode = decl.node.alias || decl.node.name; nameNode = decl.node.alias || decl.node.name;
} }
} }
@ -921,20 +998,19 @@ export class Checker extends ParseTreeWalker {
} }
if (nameNode && rule !== undefined && message) { if (nameNode && rule !== undefined && message) {
this._fileInfo.diagnosticSink.addUnusedCodeWithTextRange( this._fileInfo.diagnosticSink.addUnusedCodeWithTextRange(`'${nameNode.value}' is not accessed`, nameNode);
`'${ nameNode.value }' is not accessed`, nameNode); this._evaluator.addDiagnostic(diagnosticLevel, rule, message, nameNode);
this._evaluator.addDiagnostic(
diagnosticLevel, rule, message, nameNode);
} }
} }
// Validates that a call to isinstance or issubclass are necessary. This is a // Validates that a call to isinstance or issubclass are necessary. This is a
// common source of programming errors. // common source of programming errors.
private _validateIsInstanceCallNecessary(node: CallNode) { private _validateIsInstanceCallNecessary(node: CallNode) {
if (node.leftExpression.nodeType !== ParseNodeType.Name || if (
(node.leftExpression.value !== 'isinstance' && node.leftExpression.nodeType !== ParseNodeType.Name ||
node.leftExpression.value !== 'issubclass') || (node.leftExpression.value !== 'isinstance' && node.leftExpression.value !== 'issubclass') ||
node.arguments.length !== 2) { node.arguments.length !== 2
) {
return; return;
} }
@ -990,15 +1066,17 @@ export class Checker extends ParseTreeWalker {
// According to PEP 544, protocol classes cannot be used as the right-hand // According to PEP 544, protocol classes cannot be used as the right-hand
// argument to isinstance or issubclass. // argument to isinstance or issubclass.
if (classTypeList.some(type => ClassType.isProtocol(type))) { if (classTypeList.some(type => ClassType.isProtocol(type))) {
this._evaluator.addError(`Protocol class cannot be used in ${ callName } call`, this._evaluator.addError(
node.arguments[1].valueExpression); `Protocol class cannot be used in ${callName} call`,
node.arguments[1].valueExpression
);
} }
const finalizeFilteredTypeList = (types: Type[]): Type => { const finalizeFilteredTypeList = (types: Type[]): Type => {
return combineTypes(types); return combineTypes(types);
}; };
const filterType = (varType: ClassType): (ObjectType[] | ClassType[]) => { const filterType = (varType: ClassType): ObjectType[] | ClassType[] => {
const filteredTypes: ClassType[] = []; const filteredTypes: ClassType[] = [];
for (const filterType of classTypeList) { for (const filterType of classTypeList) {
@ -1069,14 +1147,16 @@ export class Checker extends ParseTreeWalker {
DiagnosticRule.reportUnnecessaryIsInstance, DiagnosticRule.reportUnnecessaryIsInstance,
`Unnecessary ${callName} call: '${this._evaluator.printType(arg0Type)}' ` + `Unnecessary ${callName} call: '${this._evaluator.printType(arg0Type)}' ` +
`is never ${callType} of '${this._evaluator.printType(getTestType())}'`, `is never ${callType} of '${this._evaluator.printType(getTestType())}'`,
node); node
);
} else if (isTypeSame(filteredType, arg0Type)) { } else if (isTypeSame(filteredType, arg0Type)) {
this._evaluator.addDiagnostic( this._evaluator.addDiagnostic(
this._fileInfo.diagnosticSettings.reportUnnecessaryIsInstance, this._fileInfo.diagnosticSettings.reportUnnecessaryIsInstance,
DiagnosticRule.reportUnnecessaryIsInstance, DiagnosticRule.reportUnnecessaryIsInstance,
`Unnecessary ${callName} call: '${this._evaluator.printType(arg0Type)}' ` + `Unnecessary ${callName} call: '${this._evaluator.printType(arg0Type)}' ` +
`is always ${callType} of '${this._evaluator.printType(getTestType())}'`, `is always ${callType} of '${this._evaluator.printType(getTestType())}'`,
node); node
);
} }
} }
@ -1123,8 +1203,8 @@ export class Checker extends ParseTreeWalker {
const declarations = this._evaluator.getDeclarationsForNameNode(node); const declarations = this._evaluator.getDeclarationsForNameNode(node);
let primaryDeclaration = declarations && declarations.length > 0 ? let primaryDeclaration =
declarations[declarations.length - 1] : undefined; declarations && declarations.length > 0 ? declarations[declarations.length - 1] : undefined;
if (!primaryDeclaration || primaryDeclaration.node === node) { if (!primaryDeclaration || primaryDeclaration.node === node) {
return; return;
} }
@ -1136,17 +1216,17 @@ export class Checker extends ParseTreeWalker {
let classOrModuleNode: ClassNode | ModuleNode | undefined; let classOrModuleNode: ClassNode | ModuleNode | undefined;
if (primaryDeclaration.node) { if (primaryDeclaration.node) {
classOrModuleNode = ParseTreeUtils.getEnclosingClassOrModule( classOrModuleNode = ParseTreeUtils.getEnclosingClassOrModule(primaryDeclaration.node);
primaryDeclaration.node);
} }
// If this is the name of a class, find the module or class that contains it rather // If this is the name of a class, find the module or class that contains it rather
// than constraining the use of the class name within the class itself. // than constraining the use of the class name within the class itself.
if (primaryDeclaration.node && if (
primaryDeclaration.node &&
primaryDeclaration.node.parent && primaryDeclaration.node.parent &&
primaryDeclaration.node.parent === classOrModuleNode && primaryDeclaration.node.parent === classOrModuleNode &&
classOrModuleNode.nodeType === ParseNodeType.Class) { classOrModuleNode.nodeType === ParseNodeType.Class
) {
classOrModuleNode = ParseTreeUtils.getEnclosingClassOrModule(classOrModuleNode); classOrModuleNode = ParseTreeUtils.getEnclosingClassOrModule(classOrModuleNode);
} }
@ -1166,12 +1246,16 @@ export class Checker extends ParseTreeWalker {
// If the referencing class is a subclass of the declaring class, it's // If the referencing class is a subclass of the declaring class, it's
// allowed to access a protected name. // allowed to access a protected name.
if (enclosingClassTypeInfo && if (
enclosingClassTypeInfo.decoratedType.category === TypeCategory.Class) { enclosingClassTypeInfo &&
enclosingClassTypeInfo.decoratedType.category === TypeCategory.Class
if (derivesFromClassRecursive(enclosingClassTypeInfo.decoratedType, ) {
declClassTypeInfo.decoratedType)) { if (
derivesFromClassRecursive(
enclosingClassTypeInfo.decoratedType,
declClassTypeInfo.decoratedType
)
) {
return; return;
} }
} }
@ -1186,16 +1270,17 @@ export class Checker extends ParseTreeWalker {
this._fileInfo.diagnosticSettings.reportPrivateUsage, this._fileInfo.diagnosticSettings.reportPrivateUsage,
DiagnosticRule.reportPrivateUsage, DiagnosticRule.reportPrivateUsage,
`'${nameValue}' is protected and used outside of a derived class`, `'${nameValue}' is protected and used outside of a derived class`,
node); node
);
} else { } else {
const scopeName = classOrModuleNode.nodeType === ParseNodeType.Class ? const scopeName = classOrModuleNode.nodeType === ParseNodeType.Class ? 'class' : 'module';
'class' : 'module';
this._evaluator.addDiagnostic( this._evaluator.addDiagnostic(
this._fileInfo.diagnosticSettings.reportPrivateUsage, this._fileInfo.diagnosticSettings.reportPrivateUsage,
DiagnosticRule.reportPrivateUsage, DiagnosticRule.reportPrivateUsage,
`'${nameValue}' is private and used outside of the ${scopeName} in which it is declared`, `'${nameValue}' is private and used outside of the ${scopeName} in which it is declared`,
node); node
);
} }
} }
} }
@ -1205,20 +1290,19 @@ export class Checker extends ParseTreeWalker {
// strings, and "pass" statements or ellipses. // strings, and "pass" statements or ellipses.
private _validateTypedDictClassSuite(suiteNode: SuiteNode) { private _validateTypedDictClassSuite(suiteNode: SuiteNode) {
const emitBadStatementError = (node: ParseNode) => { const emitBadStatementError = (node: ParseNode) => {
this._evaluator.addError( this._evaluator.addError(`TypedDict classes can contain only type annotations`, node);
`TypedDict classes can contain only type annotations`,
node);
}; };
suiteNode.statements.forEach(statement => { suiteNode.statements.forEach(statement => {
if (!AnalyzerNodeInfo.isCodeUnreachable(statement)) { if (!AnalyzerNodeInfo.isCodeUnreachable(statement)) {
if (statement.nodeType === ParseNodeType.StatementList) { if (statement.nodeType === ParseNodeType.StatementList) {
for (const substatement of statement.statements) { for (const substatement of statement.statements) {
if (substatement.nodeType !== ParseNodeType.TypeAnnotation && if (
substatement.nodeType !== ParseNodeType.TypeAnnotation &&
substatement.nodeType !== ParseNodeType.Ellipsis && substatement.nodeType !== ParseNodeType.Ellipsis &&
substatement.nodeType !== ParseNodeType.StringList && substatement.nodeType !== ParseNodeType.StringList &&
substatement.nodeType !== ParseNodeType.Pass) { substatement.nodeType !== ParseNodeType.Pass
) {
emitBadStatementError(substatement); emitBadStatementError(substatement);
} }
} }
@ -1247,13 +1331,18 @@ export class Checker extends ParseTreeWalker {
this._evaluator.addDiagnostic( this._evaluator.addDiagnostic(
this._fileInfo.diagnosticSettings.reportUnknownVariableType, this._fileInfo.diagnosticSettings.reportUnknownVariableType,
DiagnosticRule.reportUnknownVariableType, DiagnosticRule.reportUnknownVariableType,
`Declared return type is unknown`, node.returnTypeAnnotation); `Declared return type is unknown`,
node.returnTypeAnnotation
);
} else if (containsUnknown(declaredReturnType)) { } else if (containsUnknown(declaredReturnType)) {
this._evaluator.addDiagnostic( this._evaluator.addDiagnostic(
this._fileInfo.diagnosticSettings.reportUnknownVariableType, this._fileInfo.diagnosticSettings.reportUnknownVariableType,
DiagnosticRule.reportUnknownVariableType, DiagnosticRule.reportUnknownVariableType,
`Declared return type, '${ this._evaluator.printType(declaredReturnType) }', is partially unknown`, `Declared return type, '${this._evaluator.printType(
node.returnTypeAnnotation); declaredReturnType
)}', is partially unknown`,
node.returnTypeAnnotation
);
} }
} }
@ -1273,7 +1362,8 @@ export class Checker extends ParseTreeWalker {
if (!ParseTreeUtils.isSuiteEmpty(node.suite)) { if (!ParseTreeUtils.isSuiteEmpty(node.suite)) {
this._evaluator.addError( this._evaluator.addError(
`Function with declared type of 'NoReturn' cannot return 'None'`, `Function with declared type of 'NoReturn' cannot return 'None'`,
node.returnTypeAnnotation); node.returnTypeAnnotation
);
} }
} else if (!FunctionType.isAbstractMethod(functionType)) { } else if (!FunctionType.isAbstractMethod(functionType)) {
// Make sure that the function doesn't implicitly return None if the declared // Make sure that the function doesn't implicitly return None if the declared
@ -1288,8 +1378,10 @@ export class Checker extends ParseTreeWalker {
if (!ParseTreeUtils.isSuiteEmpty(node.suite)) { if (!ParseTreeUtils.isSuiteEmpty(node.suite)) {
this._evaluator.addError( this._evaluator.addError(
`Function with declared type of '${this._evaluator.printType(declaredReturnType)}'` + `Function with declared type of '${this._evaluator.printType(declaredReturnType)}'` +
` must return value` + diagAddendum.getString(), ` must return value` +
node.returnTypeAnnotation); diagAddendum.getString(),
node.returnTypeAnnotation
);
} }
} }
} }
@ -1300,13 +1392,16 @@ export class Checker extends ParseTreeWalker {
this._evaluator.addDiagnostic( this._evaluator.addDiagnostic(
this._fileInfo.diagnosticSettings.reportUnknownParameterType, this._fileInfo.diagnosticSettings.reportUnknownParameterType,
DiagnosticRule.reportUnknownParameterType, DiagnosticRule.reportUnknownParameterType,
`Inferred return type is unknown`, node.name); `Inferred return type is unknown`,
node.name
);
} else if (containsUnknown(inferredReturnType)) { } else if (containsUnknown(inferredReturnType)) {
this._evaluator.addDiagnostic( this._evaluator.addDiagnostic(
this._fileInfo.diagnosticSettings.reportUnknownParameterType, this._fileInfo.diagnosticSettings.reportUnknownParameterType,
DiagnosticRule.reportUnknownParameterType, DiagnosticRule.reportUnknownParameterType,
`Return type '${this._evaluator.printType(inferredReturnType)}' is partially unknown`, `Return type '${this._evaluator.printType(inferredReturnType)}' is partially unknown`,
node.name); node.name
);
} }
} }
} }
@ -1321,7 +1416,8 @@ export class Checker extends ParseTreeWalker {
this._evaluator.addError( this._evaluator.addError(
`'${name}' cannot be redeclared because parent class ` + `'${name}' cannot be redeclared because parent class ` +
`'${parentSymbol.class.details.name}' declares it as Final`, `'${parentSymbol.class.details.name}' declares it as Final`,
decl.node); decl.node
);
} }
}); });
} }
@ -1346,21 +1442,22 @@ export class Checker extends ParseTreeWalker {
const baseClassAndSymbol = getSymbolFromBaseClasses(classType, name); const baseClassAndSymbol = getSymbolFromBaseClasses(classType, name);
if (baseClassAndSymbol) { if (baseClassAndSymbol) {
const typeOfBaseClassMethod = this._evaluator.getEffectiveTypeOfSymbol( const typeOfBaseClassMethod = this._evaluator.getEffectiveTypeOfSymbol(
baseClassAndSymbol.symbol); baseClassAndSymbol.symbol
);
const diagAddendum = new DiagnosticAddendum(); const diagAddendum = new DiagnosticAddendum();
if (!this._evaluator.canOverrideMethod(typeOfBaseClassMethod, typeOfSymbol, if (!this._evaluator.canOverrideMethod(typeOfBaseClassMethod, typeOfSymbol, diagAddendum)) {
diagAddendum)) {
const decl = getLastTypedDeclaredForSymbol(symbol); const decl = getLastTypedDeclaredForSymbol(symbol);
if (decl && decl.type === DeclarationType.Function) { if (decl && decl.type === DeclarationType.Function) {
const diag = this._evaluator.addDiagnostic( const diag = this._evaluator.addDiagnostic(
this._fileInfo.diagnosticSettings.reportIncompatibleMethodOverride, this._fileInfo.diagnosticSettings.reportIncompatibleMethodOverride,
DiagnosticRule.reportIncompatibleMethodOverride, DiagnosticRule.reportIncompatibleMethodOverride,
`Method '${name}' overrides class '${baseClassAndSymbol.class.details.name}' ` + `Method '${name}' overrides class '${baseClassAndSymbol.class.details.name}' ` +
`in an incompatible manner` + diagAddendum.getString(), decl.node.name); `in an incompatible manner` +
diagAddendum.getString(),
decl.node.name
);
const origDecl = getLastTypedDeclaredForSymbol( const origDecl = getLastTypedDeclaredForSymbol(baseClassAndSymbol.symbol);
baseClassAndSymbol.symbol);
if (diag && origDecl) { if (diag && origDecl) {
diag.addRelatedInfo('Overridden method', origDecl.path, origDecl.range); diag.addRelatedInfo('Overridden method', origDecl.path, origDecl.range);
} }
@ -1374,10 +1471,10 @@ export class Checker extends ParseTreeWalker {
const diag = this._evaluator.addError( const diag = this._evaluator.addError(
`Method '${name}' cannot override final method defined ` + `Method '${name}' cannot override final method defined ` +
`in class '${baseClassAndSymbol.class.details.name}'`, `in class '${baseClassAndSymbol.class.details.name}'`,
decl.node.name); decl.node.name
);
const origDecl = getLastTypedDeclaredForSymbol( const origDecl = getLastTypedDeclaredForSymbol(baseClassAndSymbol.symbol);
baseClassAndSymbol.symbol);
if (diag && origDecl) { if (diag && origDecl) {
diag.addRelatedInfo('Final method', origDecl.path, origDecl.range); diag.addRelatedInfo('Final method', origDecl.path, origDecl.range);
} }
@ -1395,24 +1492,27 @@ export class Checker extends ParseTreeWalker {
private _validateMethod(node: FunctionNode, functionType: FunctionType, classNode: ClassNode) { private _validateMethod(node: FunctionNode, functionType: FunctionType, classNode: ClassNode) {
if (node.name && node.name.value === '__new__') { if (node.name && node.name.value === '__new__') {
// __new__ overrides should have a "cls" parameter. // __new__ overrides should have a "cls" parameter.
if (node.parameters.length === 0 || !node.parameters[0].name || if (
(node.parameters[0].name.value !== 'cls' && node.parameters.length === 0 ||
node.parameters[0].name.value !== 'mcs')) { !node.parameters[0].name ||
(node.parameters[0].name.value !== 'cls' && node.parameters[0].name.value !== 'mcs')
) {
this._evaluator.addDiagnostic( this._evaluator.addDiagnostic(
this._fileInfo.diagnosticSettings.reportSelfClsParameterName, this._fileInfo.diagnosticSettings.reportSelfClsParameterName,
DiagnosticRule.reportSelfClsParameterName, DiagnosticRule.reportSelfClsParameterName,
`The __new__ override should take a 'cls' parameter`, `The __new__ override should take a 'cls' parameter`,
node.parameters.length > 0 ? node.parameters[0] : node.name); node.parameters.length > 0 ? node.parameters[0] : node.name
);
} }
} else if (node.name && node.name.value === '__init_subclass__') { } else if (node.name && node.name.value === '__init_subclass__') {
// __init_subclass__ overrides should have a "cls" parameter. // __init_subclass__ overrides should have a "cls" parameter.
if (node.parameters.length === 0 || !node.parameters[0].name || if (node.parameters.length === 0 || !node.parameters[0].name || node.parameters[0].name.value !== 'cls') {
node.parameters[0].name.value !== 'cls') {
this._evaluator.addDiagnostic( this._evaluator.addDiagnostic(
this._fileInfo.diagnosticSettings.reportSelfClsParameterName, this._fileInfo.diagnosticSettings.reportSelfClsParameterName,
DiagnosticRule.reportSelfClsParameterName, DiagnosticRule.reportSelfClsParameterName,
`The __init_subclass__ override should take a 'cls' parameter`, `The __init_subclass__ override should take a 'cls' parameter`,
node.parameters.length > 0 ? node.parameters[0] : node.name); node.parameters.length > 0 ? node.parameters[0] : node.name
);
} }
} else if (FunctionType.isStaticMethod(functionType)) { } else if (FunctionType.isStaticMethod(functionType)) {
// Static methods should not have "self" or "cls" parameters. // Static methods should not have "self" or "cls" parameters.
@ -1423,7 +1523,8 @@ export class Checker extends ParseTreeWalker {
this._fileInfo.diagnosticSettings.reportSelfClsParameterName, this._fileInfo.diagnosticSettings.reportSelfClsParameterName,
DiagnosticRule.reportSelfClsParameterName, DiagnosticRule.reportSelfClsParameterName,
`Static methods should not take a 'self' or 'cls' parameter`, `Static methods should not take a 'self' or 'cls' parameter`,
node.parameters[0].name); node.parameters[0].name
);
} }
} }
} else if (FunctionType.isClassMethod(functionType)) { } else if (FunctionType.isClassMethod(functionType)) {
@ -1440,7 +1541,8 @@ export class Checker extends ParseTreeWalker {
this._fileInfo.diagnosticSettings.reportSelfClsParameterName, this._fileInfo.diagnosticSettings.reportSelfClsParameterName,
DiagnosticRule.reportSelfClsParameterName, DiagnosticRule.reportSelfClsParameterName,
`Class methods should take a 'cls' parameter`, `Class methods should take a 'cls' parameter`,
node.parameters.length > 0 ? node.parameters[0] : node.name); node.parameters.length > 0 ? node.parameters[0] : node.name
);
} }
} }
} else { } else {
@ -1466,9 +1568,12 @@ export class Checker extends ParseTreeWalker {
if (paramName === 'cls') { if (paramName === 'cls') {
const classTypeInfo = this._evaluator.getTypeOfClass(classNode); const classTypeInfo = this._evaluator.getTypeOfClass(classNode);
const typeType = this._evaluator.getBuiltInType(classNode, 'type'); const typeType = this._evaluator.getBuiltInType(classNode, 'type');
if (typeType && typeType.category === TypeCategory.Class && if (
classTypeInfo && classTypeInfo.classType.category === TypeCategory.Class) { typeType &&
typeType.category === TypeCategory.Class &&
classTypeInfo &&
classTypeInfo.classType.category === TypeCategory.Class
) {
if (derivesFromClassRecursive(classTypeInfo.classType, typeType)) { if (derivesFromClassRecursive(classTypeInfo.classType, typeType)) {
isLegalMetaclassName = true; isLegalMetaclassName = true;
} }
@ -1480,7 +1585,8 @@ export class Checker extends ParseTreeWalker {
this._fileInfo.diagnosticSettings.reportSelfClsParameterName, this._fileInfo.diagnosticSettings.reportSelfClsParameterName,
DiagnosticRule.reportSelfClsParameterName, DiagnosticRule.reportSelfClsParameterName,
`Instance methods should take a 'self' parameter`, `Instance methods should take a 'self' parameter`,
node.parameters.length > 0 ? node.parameters[0] : node.name); node.parameters.length > 0 ? node.parameters[0] : node.name
);
} }
} }
} }
@ -1505,14 +1611,17 @@ export class Checker extends ParseTreeWalker {
if (isNoReturnType(declaredYieldType)) { if (isNoReturnType(declaredYieldType)) {
this._evaluator.addError( this._evaluator.addError(
`Function with declared return type 'NoReturn' cannot include a yield statement`, `Function with declared return type 'NoReturn' cannot include a yield statement`,
node); node
);
} else { } else {
const diagAddendum = new DiagnosticAddendum(); const diagAddendum = new DiagnosticAddendum();
if (!this._evaluator.canAssignType(declaredYieldType, adjustedYieldType, diagAddendum)) { if (!this._evaluator.canAssignType(declaredYieldType, adjustedYieldType, diagAddendum)) {
this._evaluator.addError( this._evaluator.addError(
`Expression of type '${this._evaluator.printType(adjustedYieldType)}' cannot be assigned ` + `Expression of type '${this._evaluator.printType(adjustedYieldType)}' cannot be assigned ` +
`to yield type '${ this._evaluator.printType(declaredYieldType) }'` + diagAddendum.getString(), `to yield type '${this._evaluator.printType(declaredYieldType)}'` +
node.expression || node); diagAddendum.getString(),
node.expression || node
);
} }
} }
} }
@ -1537,7 +1646,8 @@ export class Checker extends ParseTreeWalker {
this._fileInfo.diagnosticSettings.reportDuplicateImport, this._fileInfo.diagnosticSettings.reportDuplicateImport,
DiagnosticRule.reportDuplicateImport, DiagnosticRule.reportDuplicateImport,
`'${importFromAs.name.value}' is imported more than once`, `'${importFromAs.name.value}' is imported more than once`,
importFromAs.name); importFromAs.name
);
} else { } else {
symbolMap.set(importFromAs.name.value, importFromAs); symbolMap.set(importFromAs.name.value, importFromAs);
} }
@ -1552,7 +1662,8 @@ export class Checker extends ParseTreeWalker {
this._fileInfo.diagnosticSettings.reportDuplicateImport, this._fileInfo.diagnosticSettings.reportDuplicateImport,
DiagnosticRule.reportDuplicateImport, DiagnosticRule.reportDuplicateImport,
`'${importStatement.moduleName}' is imported more than once`, `'${importStatement.moduleName}' is imported more than once`,
importStatement.subnode); importStatement.subnode
);
} else { } else {
importModuleMap.set(importStatement.moduleName, importStatement.subnode); importModuleMap.set(importStatement.moduleName, importStatement.subnode);
} }

3
server/src/analyzer/circularDependency.ts
View File

@ -32,8 +32,7 @@ export class CircularDependency {
}); });
if (firstIndex !== 0) { if (firstIndex !== 0) {
this._paths = this._paths.slice(firstIndex).concat( this._paths = this._paths.slice(firstIndex).concat(this._paths.slice(0, firstIndex));
this._paths.slice(0, firstIndex));
} }
} }

10
server/src/analyzer/codeFlow.ts
View File

@ -14,8 +14,14 @@
*/ */
import { assert } from '../common/debug'; import { assert } from '../common/debug';
import { CallNode, ExpressionNode, ImportFromNode, MemberAccessNode, NameNode, import {
ParseNodeType } from '../parser/parseNodes'; CallNode,
ExpressionNode,
ImportFromNode,
MemberAccessNode,
NameNode,
ParseNodeType
} from '../parser/parseNodes';
export enum FlowFlags { export enum FlowFlags {
Unreachable = 1 << 0, // Unreachable code Unreachable = 1 << 0, // Unreachable code

18
server/src/analyzer/commentUtils.ts
View File

@ -8,14 +8,22 @@
* or other directives from them. * or other directives from them.
*/ */
import { cloneDiagnosticSettings, DiagnosticLevel, DiagnosticSettings, import {
getBooleanDiagnosticSettings, getDiagLevelSettings, getStrictDiagnosticSettings } from '../common/configOptions'; cloneDiagnosticSettings,
DiagnosticLevel,
DiagnosticSettings,
getBooleanDiagnosticSettings,
getDiagLevelSettings,
getStrictDiagnosticSettings
} from '../common/configOptions';
import { TextRangeCollection } from '../common/textRangeCollection'; import { TextRangeCollection } from '../common/textRangeCollection';
import { Token } from '../parser/tokenizerTypes'; import { Token } from '../parser/tokenizerTypes';
export function getFileLevelDirectives(tokens: TextRangeCollection<Token>, export function getFileLevelDirectives(
defaultSettings: DiagnosticSettings, useStrict: boolean): DiagnosticSettings { tokens: TextRangeCollection<Token>,
defaultSettings: DiagnosticSettings,
useStrict: boolean
): DiagnosticSettings {
let settings = cloneDiagnosticSettings(defaultSettings); let settings = cloneDiagnosticSettings(defaultSettings);
if (useStrict) { if (useStrict) {

32
server/src/analyzer/declaration.ts
View File

@ -10,10 +10,23 @@
*/ */
import { Range } from '../common/textRange'; import { Range } from '../common/textRange';
import { ClassNode, ExpressionNode, FunctionNode, ImportAsNode, import {
ImportFromAsNode, ImportFromNode, ModuleNode, NameNode, ParameterNode, ClassNode,
ParseNode, ReturnNode, StringListNode, TypeAnnotationNode, YieldFromNode, ExpressionNode,
YieldNode } from '../parser/parseNodes'; FunctionNode,
ImportAsNode,
ImportFromAsNode,
ImportFromNode,
ModuleNode,
NameNode,
ParameterNode,
ParseNode,
ReturnNode,
StringListNode,
TypeAnnotationNode,
YieldFromNode,
YieldNode
} from '../parser/parseNodes';
export const enum DeclarationType { export const enum DeclarationType {
Intrinsic, Intrinsic,
@ -132,6 +145,11 @@ export interface ModuleLoaderActions {
implicitImports?: Map<string, ModuleLoaderActions>; implicitImports?: Map<string, ModuleLoaderActions>;
} }
export type Declaration = IntrinsicDeclaration | ClassDeclaration | export type Declaration =
SpecialBuiltInClassDeclaration | FunctionDeclaration | ParameterDeclaration | | IntrinsicDeclaration
VariableDeclaration | AliasDeclaration; | ClassDeclaration
| SpecialBuiltInClassDeclaration
| FunctionDeclaration
| ParameterDeclaration
| VariableDeclaration
| AliasDeclaration;

6
server/src/analyzer/declarationUtils.ts
View File

@ -37,8 +37,10 @@ export function areDeclarationsSame(decl1: Declaration, decl2: Declaration): boo
return false; return false;
} }
if (decl1.range.start.line !== decl2.range.start.line || if (
decl1.range.start.character !== decl2.range.start.character) { decl1.range.start.line !== decl2.range.start.line ||
decl1.range.start.character !== decl2.range.start.character
) {
return false; return false;
} }

370
server/src/analyzer/importResolver.ts
View File

@ -10,9 +10,16 @@
import { ConfigOptions, ExecutionEnvironment } from '../common/configOptions'; import { ConfigOptions, ExecutionEnvironment } from '../common/configOptions';
import { import {
combinePaths, ensureTrailingDirectorySeparator, getDirectoryPath, combinePaths,
getFileExtension, getFileSystemEntries, getPathComponents, isDirectory, ensureTrailingDirectorySeparator,
isFile, stripFileExtension, stripTrailingDirectorySeparator getDirectoryPath,
getFileExtension,
getFileSystemEntries,
getPathComponents,
isDirectory,
isFile,
stripFileExtension,
stripTrailingDirectorySeparator
} from '../common/pathUtils'; } from '../common/pathUtils';
import { versionToString } from '../common/pythonVersion'; import { versionToString } from '../common/pythonVersion';
import * as StringUtils from '../common/stringUtils'; import * as StringUtils from '../common/stringUtils';
@ -56,16 +63,22 @@ export class ImportResolver {
// Resolves the import and returns the path if it exists, otherwise // Resolves the import and returns the path if it exists, otherwise
// returns undefined. // returns undefined.
resolveImport(sourceFilePath: string, execEnv: ExecutionEnvironment, resolveImport(
moduleDescriptor: ImportedModuleDescriptor): ImportResult { sourceFilePath: string,
execEnv: ExecutionEnvironment,
moduleDescriptor: ImportedModuleDescriptor
): ImportResult {
const importName = this._formatImportName(moduleDescriptor); const importName = this._formatImportName(moduleDescriptor);
const importFailureInfo: string[] = []; const importFailureInfo: string[] = [];
// Is it a relative import? // Is it a relative import?
if (moduleDescriptor.leadingDots > 0) { if (moduleDescriptor.leadingDots > 0) {
const relativeImport = this._resolveRelativeImport(sourceFilePath, const relativeImport = this._resolveRelativeImport(
moduleDescriptor, importName, importFailureInfo); sourceFilePath,
moduleDescriptor,
importName,
importFailureInfo
);
if (relativeImport) { if (relativeImport) {
relativeImport.isRelative = true; relativeImport.isRelative = true;
@ -73,47 +86,53 @@ export class ImportResolver {
} }
} else { } else {
// Is it already cached? // Is it already cached?
const cachedResults = this._lookUpResultsInCache(execEnv, importName, const cachedResults = this._lookUpResultsInCache(execEnv, importName, moduleDescriptor.importedSymbols);
moduleDescriptor.importedSymbols);
if (cachedResults) { if (cachedResults) {
return cachedResults; return cachedResults;
} }
// First check for a typeshed file. // First check for a typeshed file.
if (moduleDescriptor.nameParts.length > 0) { if (moduleDescriptor.nameParts.length > 0) {
const builtInImport = this._findTypeshedPath(execEnv, moduleDescriptor, const builtInImport = this._findTypeshedPath(
importName, true, importFailureInfo); execEnv,
moduleDescriptor,
importName,
true,
importFailureInfo
);
if (builtInImport) { if (builtInImport) {
builtInImport.isTypeshedFile = true; builtInImport.isTypeshedFile = true;
return this._addResultsToCache(execEnv, importName, builtInImport, return this._addResultsToCache(
moduleDescriptor.importedSymbols); execEnv,
importName,
builtInImport,
moduleDescriptor.importedSymbols
);
} }
} }
let bestResultSoFar: ImportResult | undefined; let bestResultSoFar: ImportResult | undefined;
// Look for it in the root directory of the execution environment. // Look for it in the root directory of the execution environment.
importFailureInfo.push(`Looking in root directory of execution environment ` + importFailureInfo.push(`Looking in root directory of execution environment ` + `'${execEnv.root}'`);
`'${ execEnv.root }'`); let localImport = this.resolveAbsoluteImport(execEnv.root, moduleDescriptor, importName, importFailureInfo);
let localImport = this.resolveAbsoluteImport(
execEnv.root, moduleDescriptor, importName, importFailureInfo);
if (localImport && localImport.isImportFound) { if (localImport && localImport.isImportFound) {
return this._addResultsToCache(execEnv, importName, localImport, return this._addResultsToCache(execEnv, importName, localImport, moduleDescriptor.importedSymbols);
moduleDescriptor.importedSymbols);
} }
bestResultSoFar = localImport; bestResultSoFar = localImport;
for (const extraPath of execEnv.extraPaths) { for (const extraPath of execEnv.extraPaths) {
importFailureInfo.push(`Looking in extraPath '${extraPath}'`); importFailureInfo.push(`Looking in extraPath '${extraPath}'`);
localImport = this.resolveAbsoluteImport(extraPath, moduleDescriptor, localImport = this.resolveAbsoluteImport(extraPath, moduleDescriptor, importName, importFailureInfo);
importName, importFailureInfo);
if (localImport && localImport.isImportFound) { if (localImport && localImport.isImportFound) {
return this._addResultsToCache(execEnv, importName, localImport, return this._addResultsToCache(execEnv, importName, localImport, moduleDescriptor.importedSymbols);
moduleDescriptor.importedSymbols);
} }
if (localImport && (bestResultSoFar === undefined || if (
localImport.resolvedPaths.length > bestResultSoFar.resolvedPaths.length)) { localImport &&
(bestResultSoFar === undefined ||
localImport.resolvedPaths.length > bestResultSoFar.resolvedPaths.length)
) {
bestResultSoFar = localImport; bestResultSoFar = localImport;
} }
} }
@ -122,24 +141,36 @@ export class ImportResolver {
if (this._configOptions.typingsPath) { if (this._configOptions.typingsPath) {
importFailureInfo.push(`Looking in typingsPath '${this._configOptions.typingsPath}'`); importFailureInfo.push(`Looking in typingsPath '${this._configOptions.typingsPath}'`);
const typingsImport = this.resolveAbsoluteImport( const typingsImport = this.resolveAbsoluteImport(
this._configOptions.typingsPath, moduleDescriptor, importName, importFailureInfo); this._configOptions.typingsPath,
moduleDescriptor,
importName,
importFailureInfo
);
if (typingsImport && typingsImport.isImportFound) { if (typingsImport && typingsImport.isImportFound) {
// We will treat typings files as "local" rather than "third party". // We will treat typings files as "local" rather than "third party".
typingsImport.importType = ImportType.Local; typingsImport.importType = ImportType.Local;
typingsImport.isLocalTypingsFile = true; typingsImport.isLocalTypingsFile = true;
return this._addResultsToCache(execEnv, importName, typingsImport, return this._addResultsToCache(
moduleDescriptor.importedSymbols); execEnv,
importName,
typingsImport,
moduleDescriptor.importedSymbols
);
} }
} }
// Check for a typeshed file. // Check for a typeshed file.
importFailureInfo.push(`Looking for typeshed path`); importFailureInfo.push(`Looking for typeshed path`);
const typeshedImport = this._findTypeshedPath(execEnv, moduleDescriptor, const typeshedImport = this._findTypeshedPath(
importName, false, importFailureInfo); execEnv,
moduleDescriptor,
importName,
false,
importFailureInfo
);
if (typeshedImport) { if (typeshedImport) {
typeshedImport.isTypeshedFile = true; typeshedImport.isTypeshedFile = true;
return this._addResultsToCache(execEnv, importName, typeshedImport, return this._addResultsToCache(execEnv, importName, typeshedImport, moduleDescriptor.importedSymbols);
moduleDescriptor.importedSymbols);
} }
// Look for the import in the list of third-party packages. // Look for the import in the list of third-party packages.
@ -150,21 +181,33 @@ export class ImportResolver {
// use tricks to populate their package namespaces. // use tricks to populate their package namespaces.
importFailureInfo.push(`Looking in python search path '${searchPath}'`); importFailureInfo.push(`Looking in python search path '${searchPath}'`);
const thirdPartyImport = this.resolveAbsoluteImport( const thirdPartyImport = this.resolveAbsoluteImport(
searchPath, moduleDescriptor, importName, importFailureInfo, searchPath,
true, true, true); moduleDescriptor,
importName,
importFailureInfo,
true,
true,
true
);
if (thirdPartyImport) { if (thirdPartyImport) {
thirdPartyImport.importType = ImportType.ThirdParty; thirdPartyImport.importType = ImportType.ThirdParty;
if (thirdPartyImport.isImportFound && thirdPartyImport.isStubFile) { if (thirdPartyImport.isImportFound && thirdPartyImport.isStubFile) {
return this._addResultsToCache(execEnv, importName, return this._addResultsToCache(
thirdPartyImport, moduleDescriptor.importedSymbols); execEnv,
importName,
thirdPartyImport,
moduleDescriptor.importedSymbols
);
} }
// We did not find it, or we did and it's not from a // We did not find it, or we did and it's not from a
// stub, so give chance for resolveImportEx to find // stub, so give chance for resolveImportEx to find
// one from a stub. // one from a stub.
if (bestResultSoFar === undefined || if (
thirdPartyImport.resolvedPaths.length > bestResultSoFar.resolvedPaths.length) { bestResultSoFar === undefined ||
thirdPartyImport.resolvedPaths.length > bestResultSoFar.resolvedPaths.length
) {
bestResultSoFar = thirdPartyImport; bestResultSoFar = thirdPartyImport;
} }
} }
@ -173,17 +216,21 @@ export class ImportResolver {
importFailureInfo.push('No python interpreter search path'); importFailureInfo.push('No python interpreter search path');
} }
const extraResults = this.resolveImportEx(sourceFilePath, execEnv, moduleDescriptor, importName, importFailureInfo); const extraResults = this.resolveImportEx(
sourceFilePath,
execEnv,
moduleDescriptor,
importName,
importFailureInfo
);
if (extraResults !== undefined) { if (extraResults !== undefined) {
return this._addResultsToCache(execEnv, importName, extraResults, return this._addResultsToCache(execEnv, importName, extraResults, moduleDescriptor.importedSymbols);
moduleDescriptor.importedSymbols);
} }
// We weren't able to find an exact match, so return the best // We weren't able to find an exact match, so return the best
// partial match. // partial match.
if (bestResultSoFar) { if (bestResultSoFar) {
return this._addResultsToCache(execEnv, importName, bestResultSoFar, return this._addResultsToCache(execEnv, importName, bestResultSoFar, moduleDescriptor.importedSymbols);
moduleDescriptor.importedSymbols);
} }
} }
@ -205,53 +252,64 @@ export class ImportResolver {
// Intended to be overridden by subclasses to provide additional stub // Intended to be overridden by subclasses to provide additional stub
// resolving capabilities. Return undefined if no stubs were found for // resolving capabilities. Return undefined if no stubs were found for
// this import. // this import.
protected resolveImportEx(sourceFilePath: string, execEnv: ExecutionEnvironment, protected resolveImportEx(
moduleDescriptor: ImportedModuleDescriptor, importName: string, sourceFilePath: string,
importFailureInfo: string[] = []): ImportResult | undefined { execEnv: ExecutionEnvironment,
moduleDescriptor: ImportedModuleDescriptor,
importName: string,
importFailureInfo: string[] = []
): ImportResult | undefined {
return undefined; return undefined;
} }
getCompletionSuggestions(sourceFilePath: string, execEnv: ExecutionEnvironment, getCompletionSuggestions(
moduleDescriptor: ImportedModuleDescriptor, similarityLimit: number): string[] { sourceFilePath: string,
execEnv: ExecutionEnvironment,
moduleDescriptor: ImportedModuleDescriptor,
similarityLimit: number
): string[] {
const importFailureInfo: string[] = []; const importFailureInfo: string[] = [];
const suggestions: string[] = []; const suggestions: string[] = [];
// Is it a relative import? // Is it a relative import?
if (moduleDescriptor.leadingDots > 0) { if (moduleDescriptor.leadingDots > 0) {
this._getCompletionSuggestionsRelative(sourceFilePath, this._getCompletionSuggestionsRelative(sourceFilePath, moduleDescriptor, suggestions, similarityLimit);
moduleDescriptor, suggestions, similarityLimit);
} else { } else {
// First check for a typeshed file. // First check for a typeshed file.
if (moduleDescriptor.nameParts.length > 0) { if (moduleDescriptor.nameParts.length > 0) {
this._getCompletionSuggestionsTypeshedPath(execEnv, moduleDescriptor, this._getCompletionSuggestionsTypeshedPath(
true, suggestions, similarityLimit); execEnv,
moduleDescriptor,
true,
suggestions,
similarityLimit
);
} }
// Look for it in the root directory of the execution environment. // Look for it in the root directory of the execution environment.
this._getCompletionSuggestionsAbsolute(execEnv.root, this._getCompletionSuggestionsAbsolute(execEnv.root, moduleDescriptor, suggestions, similarityLimit);
moduleDescriptor, suggestions, similarityLimit);
for (const extraPath of execEnv.extraPaths) { for (const extraPath of execEnv.extraPaths) {
this._getCompletionSuggestionsAbsolute(extraPath, moduleDescriptor, this._getCompletionSuggestionsAbsolute(extraPath, moduleDescriptor, suggestions, similarityLimit);
suggestions, similarityLimit);
} }
// Check for a typings file. // Check for a typings file.
if (this._configOptions.typingsPath) { if (this._configOptions.typingsPath) {
this._getCompletionSuggestionsAbsolute(this._configOptions.typingsPath, this._getCompletionSuggestionsAbsolute(
moduleDescriptor, suggestions, similarityLimit); this._configOptions.typingsPath,
moduleDescriptor,
suggestions,
similarityLimit
);
} }
// Check for a typeshed file. // Check for a typeshed file.
this._getCompletionSuggestionsTypeshedPath(execEnv, moduleDescriptor, this._getCompletionSuggestionsTypeshedPath(execEnv, moduleDescriptor, false, suggestions, similarityLimit);
false, suggestions, similarityLimit);
// Look for the import in the list of third-party packages. // Look for the import in the list of third-party packages.
const pythonSearchPaths = this._getPythonSearchPaths(execEnv, importFailureInfo); const pythonSearchPaths = this._getPythonSearchPaths(execEnv, importFailureInfo);
for (const searchPath of pythonSearchPaths) { for (const searchPath of pythonSearchPaths) {
this._getCompletionSuggestionsAbsolute(searchPath, this._getCompletionSuggestionsAbsolute(searchPath, moduleDescriptor, suggestions, similarityLimit);
moduleDescriptor, suggestions, similarityLimit);
} }
} }
@ -292,8 +350,7 @@ export class ImportResolver {
// Check for a typings file. // Check for a typings file.
if (this._configOptions.typingsPath) { if (this._configOptions.typingsPath) {
const candidateModuleName = this._getModuleNameFromPath( const candidateModuleName = this._getModuleNameFromPath(this._configOptions.typingsPath, filePath);
this._configOptions.typingsPath, filePath);
// Does this candidate look better than the previous best module name? // Does this candidate look better than the previous best module name?
// We'll always try to use the shortest version. // We'll always try to use the shortest version.
@ -337,9 +394,11 @@ export class ImportResolver {
return { moduleName: '', importType: ImportType.Local }; return { moduleName: '', importType: ImportType.Local };
} }
private _lookUpResultsInCache(execEnv: ExecutionEnvironment, importName: string, private _lookUpResultsInCache(
importedSymbols: string[] | undefined) { execEnv: ExecutionEnvironment,
importName: string,
importedSymbols: string[] | undefined
) {
const cacheForExecEnv = this._cachedImportResults.get(execEnv.root); const cacheForExecEnv = this._cachedImportResults.get(execEnv.root);
if (!cacheForExecEnv) { if (!cacheForExecEnv) {
return undefined; return undefined;
@ -353,9 +412,12 @@ export class ImportResolver {
return this._filterImplicitImports(cachedEntry, importedSymbols); return this._filterImplicitImports(cachedEntry, importedSymbols);
} }
private _addResultsToCache(execEnv: ExecutionEnvironment, importName: string, private _addResultsToCache(
importResult: ImportResult, importedSymbols: string[] | undefined) { execEnv: ExecutionEnvironment,
importName: string,
importResult: ImportResult,
importedSymbols: string[] | undefined
) {
let cacheForExecEnv = this._cachedImportResults.get(execEnv.root); let cacheForExecEnv = this._cachedImportResults.get(execEnv.root);
if (!cacheForExecEnv) { if (!cacheForExecEnv) {
cacheForExecEnv = new Map<string, ImportResult>(); cacheForExecEnv = new Map<string, ImportResult>();
@ -367,9 +429,11 @@ export class ImportResolver {
return this._filterImplicitImports(importResult, importedSymbols); return this._filterImplicitImports(importResult, importedSymbols);
} }
private _getModuleNameFromPath(containerPath: string, filePath: string, private _getModuleNameFromPath(
stripTopContainerDir = false): string | undefined { containerPath: string,
filePath: string,
stripTopContainerDir = false
): string | undefined {
containerPath = ensureTrailingDirectorySeparator(containerPath); containerPath = ensureTrailingDirectorySeparator(containerPath);
let filePathWithoutExtension = stripFileExtension(filePath); let filePathWithoutExtension = stripFileExtension(filePath);
@ -399,23 +463,32 @@ export class ImportResolver {
return parts.join('.'); return parts.join('.');
} }
private _getPythonSearchPaths(execEnv: ExecutionEnvironment, private _getPythonSearchPaths(execEnv: ExecutionEnvironment, importFailureInfo: string[]) {
importFailureInfo: string[]) {
const cacheKey = execEnv.venv ? execEnv.venv : '<default>'; const cacheKey = execEnv.venv ? execEnv.venv : '<default>';
// Find the site packages for the configured virtual environment. // Find the site packages for the configured virtual environment.
if (!this._cachedPythonSearchPaths.has(cacheKey)) { if (!this._cachedPythonSearchPaths.has(cacheKey)) {
this._cachedPythonSearchPaths.set(cacheKey, PythonPathUtils.findPythonSearchPaths( this._cachedPythonSearchPaths.set(
this.fileSystem, this._configOptions, execEnv.venv, importFailureInfo) || []); cacheKey,
PythonPathUtils.findPythonSearchPaths(
this.fileSystem,
this._configOptions,
execEnv.venv,
importFailureInfo
) || []
);
} }
return this._cachedPythonSearchPaths.get(cacheKey)!; return this._cachedPythonSearchPaths.get(cacheKey)!;
} }
private _findTypeshedPath(execEnv: ExecutionEnvironment, moduleDescriptor: ImportedModuleDescriptor, private _findTypeshedPath(
importName: string, isStdLib: boolean, importFailureInfo: string[]): ImportResult | undefined { execEnv: ExecutionEnvironment,
moduleDescriptor: ImportedModuleDescriptor,
importName: string,
isStdLib: boolean,
importFailureInfo: string[]
): ImportResult | undefined {
importFailureInfo.push(`Looking for typeshed ${isStdLib ? 'stdlib' : 'third_party'} path`); importFailureInfo.push(`Looking for typeshed ${isStdLib ? 'stdlib' : 'third_party'} path`);
const typeshedPath = this._getTypeshedPath(isStdLib, execEnv, importFailureInfo); const typeshedPath = this._getTypeshedPath(isStdLib, execEnv, importFailureInfo);
@ -424,16 +497,20 @@ export class ImportResolver {
} }
const pythonVersion = execEnv.pythonVersion; const pythonVersion = execEnv.pythonVersion;
let minorVersion = pythonVersion & 0xFF; let minorVersion = pythonVersion & 0xff;
// Search for module starting at "3.x" down to "3.1", then "3", then "2and3". // Search for module starting at "3.x" down to "3.1", then "3", then "2and3".
while (true) { while (true) {
const pythonVersionString = minorVersion > 0 ? versionToString(0x300 + minorVersion) : const pythonVersionString =
minorVersion === 0 ? '3' : '2and3'; minorVersion > 0 ? versionToString(0x300 + minorVersion) : minorVersion === 0 ? '3' : '2and3';
const testPath = combinePaths(typeshedPath, pythonVersionString); const testPath = combinePaths(typeshedPath, pythonVersionString);
if (this.fileSystem.existsSync(testPath)) { if (this.fileSystem.existsSync(testPath)) {
const importInfo = this.resolveAbsoluteImport(testPath, moduleDescriptor, const importInfo = this.resolveAbsoluteImport(
importName, importFailureInfo); testPath,
moduleDescriptor,
importName,
importFailureInfo
);
if (importInfo && importInfo.isImportFound) { if (importInfo && importInfo.isImportFound) {
importInfo.importType = isStdLib ? ImportType.BuiltIn : ImportType.ThirdParty; importInfo.importType = isStdLib ? ImportType.BuiltIn : ImportType.ThirdParty;
return importInfo; return importInfo;
@ -451,10 +528,13 @@ export class ImportResolver {
return undefined; return undefined;
} }
private _getCompletionSuggestionsTypeshedPath(execEnv: ExecutionEnvironment, private _getCompletionSuggestionsTypeshedPath(
moduleDescriptor: ImportedModuleDescriptor, isStdLib: boolean, execEnv: ExecutionEnvironment,
suggestions: string[], similarityLimit: number) { moduleDescriptor: ImportedModuleDescriptor,
isStdLib: boolean,
suggestions: string[],
similarityLimit: number
) {
const importFailureInfo: string[] = []; const importFailureInfo: string[] = [];
const typeshedPath = this._getTypeshedPath(isStdLib, execEnv, importFailureInfo); const typeshedPath = this._getTypeshedPath(isStdLib, execEnv, importFailureInfo);
if (!typeshedPath) { if (!typeshedPath) {
@ -462,16 +542,15 @@ export class ImportResolver {
} }
const pythonVersion = execEnv.pythonVersion; const pythonVersion = execEnv.pythonVersion;
let minorVersion = pythonVersion & 0xFF; let minorVersion = pythonVersion & 0xff;
// Search for module starting at "3.x" down to "3.1", then "3", then "2and3". // Search for module starting at "3.x" down to "3.1", then "3", then "2and3".
while (true) { while (true) {
const pythonVersionString = minorVersion > 0 ? versionToString(0x300 + minorVersion) : const pythonVersionString =
minorVersion === 0 ? '3' : '2and3'; minorVersion > 0 ? versionToString(0x300 + minorVersion) : minorVersion === 0 ? '3' : '2and3';
const testPath = combinePaths(typeshedPath, pythonVersionString); const testPath = combinePaths(typeshedPath, pythonVersionString);
if (this.fileSystem.existsSync(testPath)) { if (this.fileSystem.existsSync(testPath)) {
this._getCompletionSuggestionsAbsolute(testPath, moduleDescriptor, this._getCompletionSuggestionsAbsolute(testPath, moduleDescriptor, suggestions, similarityLimit);
suggestions, similarityLimit);
} }
// We use -1 to indicate "2and3", which is searched after "3.0". // We use -1 to indicate "2and3", which is searched after "3.0".
@ -482,9 +561,7 @@ export class ImportResolver {
} }
} }
private _getTypeshedPath(isStdLib: boolean, execEnv: ExecutionEnvironment, private _getTypeshedPath(isStdLib: boolean, execEnv: ExecutionEnvironment, importFailureInfo: string[]) {
importFailureInfo: string[]) {
// See if we have it cached. // See if we have it cached.
if (isStdLib) { if (isStdLib) {
if (this._cachedTypeshedStdLibPath !== undefined) { if (this._cachedTypeshedStdLibPath !== undefined) {
@ -502,14 +579,20 @@ export class ImportResolver {
// python search paths, then in the typeshed-fallback directory. // python search paths, then in the typeshed-fallback directory.
if (this._configOptions.typeshedPath) { if (this._configOptions.typeshedPath) {
const possibleTypeshedPath = this._configOptions.typeshedPath; const possibleTypeshedPath = this._configOptions.typeshedPath;
if (this.fileSystem.existsSync(possibleTypeshedPath) && isDirectory(this.fileSystem, possibleTypeshedPath)) { if (
this.fileSystem.existsSync(possibleTypeshedPath) &&
isDirectory(this.fileSystem, possibleTypeshedPath)
) {
typeshedPath = possibleTypeshedPath; typeshedPath = possibleTypeshedPath;
} }
} else { } else {
const pythonSearchPaths = this._getPythonSearchPaths(execEnv, importFailureInfo); const pythonSearchPaths = this._getPythonSearchPaths(execEnv, importFailureInfo);
for (const searchPath of pythonSearchPaths) { for (const searchPath of pythonSearchPaths) {
const possibleTypeshedPath = combinePaths(searchPath, 'typeshed'); const possibleTypeshedPath = combinePaths(searchPath, 'typeshed');
if (this.fileSystem.existsSync(possibleTypeshedPath) && isDirectory(this.fileSystem, possibleTypeshedPath)) { if (
this.fileSystem.existsSync(possibleTypeshedPath) &&
isDirectory(this.fileSystem, possibleTypeshedPath)
) {
typeshedPath = possibleTypeshedPath; typeshedPath = possibleTypeshedPath;
break; break;
} }
@ -537,10 +620,12 @@ export class ImportResolver {
return typeshedPath; return typeshedPath;
} }
private _resolveRelativeImport(sourceFilePath: string, private _resolveRelativeImport(
moduleDescriptor: ImportedModuleDescriptor, importName: string, sourceFilePath: string,
importFailureInfo: string[]): ImportResult | undefined { moduleDescriptor: ImportedModuleDescriptor,
importName: string,
importFailureInfo: string[]
): ImportResult | undefined {
importFailureInfo.push('Attempting to resolve relative import'); importFailureInfo.push('Attempting to resolve relative import');
// Determine which search path this file is part of. // Determine which search path this file is part of.
@ -554,8 +639,7 @@ export class ImportResolver {
} }
// Now try to match the module parts from the current directory location. // Now try to match the module parts from the current directory location.
const absImport = this.resolveAbsoluteImport(curDir, moduleDescriptor, const absImport = this.resolveAbsoluteImport(curDir, moduleDescriptor, importName, importFailureInfo);
importName, importFailureInfo);
if (!absImport) { if (!absImport) {
return undefined; return undefined;
} }
@ -563,10 +647,12 @@ export class ImportResolver {
return this._filterImplicitImports(absImport, moduleDescriptor.importedSymbols); return this._filterImplicitImports(absImport, moduleDescriptor.importedSymbols);
} }
private _getCompletionSuggestionsRelative(sourceFilePath: string, private _getCompletionSuggestionsRelative(
moduleDescriptor: ImportedModuleDescriptor, suggestions: string[], sourceFilePath: string,
similarityLimit: number) { moduleDescriptor: ImportedModuleDescriptor,
suggestions: string[],
similarityLimit: number
) {
// Determine which search path this file is part of. // Determine which search path this file is part of.
let curDir = getDirectoryPath(sourceFilePath); let curDir = getDirectoryPath(sourceFilePath);
for (let i = 1; i < moduleDescriptor.leadingDots; i++) { for (let i = 1; i < moduleDescriptor.leadingDots; i++) {
@ -577,16 +663,20 @@ export class ImportResolver {
} }
// Now try to match the module parts from the current directory location. // Now try to match the module parts from the current directory location.
this._getCompletionSuggestionsAbsolute(curDir, moduleDescriptor, this._getCompletionSuggestionsAbsolute(curDir, moduleDescriptor, suggestions, similarityLimit);
suggestions, similarityLimit);
} }
// Follows import resolution algorithm defined in PEP-420: // Follows import resolution algorithm defined in PEP-420:
// https://www.python.org/dev/peps/pep-0420/ // https://www.python.org/dev/peps/pep-0420/
protected resolveAbsoluteImport(rootPath: string, moduleDescriptor: ImportedModuleDescriptor, protected resolveAbsoluteImport(
importName: string, importFailureInfo: string[], allowPartial = false, rootPath: string,
allowPydFile = false, allowStubsFolder = false): ImportResult | undefined { moduleDescriptor: ImportedModuleDescriptor,
importName: string,
importFailureInfo: string[],
allowPartial = false,
allowPydFile = false,
allowStubsFolder = false
): ImportResult | undefined {
importFailureInfo.push(`Attempting to resolve using root path '${rootPath}'`); importFailureInfo.push(`Attempting to resolve using root path '${rootPath}'`);
// Starting at the specified path, walk the file system to find the // Starting at the specified path, walk the file system to find the
@ -611,7 +701,11 @@ export class ImportResolver {
} else if (this.fileSystem.existsSync(pyFilePath) && isFile(this.fileSystem, pyFilePath)) { } else if (this.fileSystem.existsSync(pyFilePath) && isFile(this.fileSystem, pyFilePath)) {
importFailureInfo.push(`Resolved import with file '${pyFilePath}'`); importFailureInfo.push(`Resolved import with file '${pyFilePath}'`);
resolvedPaths.push(pyFilePath); resolvedPaths.push(pyFilePath);
} else if (allowPydFile && this.fileSystem.existsSync(pydFilePath) && isFile(this.fileSystem, pydFilePath)) { } else if (
allowPydFile &&
this.fileSystem.existsSync(pydFilePath) &&
isFile(this.fileSystem, pydFilePath)
) {
importFailureInfo.push(`Resolved import with file '${pydFilePath}'`); importFailureInfo.push(`Resolved import with file '${pydFilePath}'`);
resolvedPaths.push(pydFilePath); resolvedPaths.push(pydFilePath);
isPydFile = true; isPydFile = true;
@ -634,7 +728,8 @@ export class ImportResolver {
// the string '-stubs' to its top-level directory name. We'll // the string '-stubs' to its top-level directory name. We'll
// look there first. // look there first.
const stubsDirPath = dirPath + '-stubs'; const stubsDirPath = dirPath + '-stubs';
foundDirectory = this.fileSystem.existsSync(stubsDirPath) && isDirectory(this.fileSystem, stubsDirPath); foundDirectory =
this.fileSystem.existsSync(stubsDirPath) && isDirectory(this.fileSystem, stubsDirPath);
if (foundDirectory) { if (foundDirectory) {
dirPath = stubsDirPath; dirPath = stubsDirPath;
} }
@ -692,7 +787,11 @@ export class ImportResolver {
} else if (this.fileSystem.existsSync(pyFilePath) && isFile(this.fileSystem, pyFilePath)) { } else if (this.fileSystem.existsSync(pyFilePath) && isFile(this.fileSystem, pyFilePath)) {
importFailureInfo.push(`Resolved import with file '${pyFilePath}'`); importFailureInfo.push(`Resolved import with file '${pyFilePath}'`);
resolvedPaths.push(pyFilePath); resolvedPaths.push(pyFilePath);
} else if (allowPydFile && this.fileSystem.existsSync(pydFilePath) && isFile(this.fileSystem, pydFilePath)) { } else if (
allowPydFile &&
this.fileSystem.existsSync(pydFilePath) &&
isFile(this.fileSystem, pydFilePath)
) {
importFailureInfo.push(`Resolved import with file '${pydFilePath}'`); importFailureInfo.push(`Resolved import with file '${pydFilePath}'`);
resolvedPaths.push(pydFilePath); resolvedPaths.push(pydFilePath);
if (isLastPart) { if (isLastPart) {
@ -706,8 +805,9 @@ export class ImportResolver {
isNamespacePackage = true; isNamespacePackage = true;
} }
} else { } else {
importFailureInfo.push(`Did not find file '${ pyiFilePath }',` + importFailureInfo.push(
` '${ pyFilePath }' or '${ pydFilePath }'`); `Did not find file '${pyiFilePath}',` + ` '${pyFilePath}' or '${pydFilePath}'`
);
} }
break; break;
} }
@ -739,10 +839,12 @@ export class ImportResolver {
}; };
} }
private _getCompletionSuggestionsAbsolute(rootPath: string, private _getCompletionSuggestionsAbsolute(
moduleDescriptor: ImportedModuleDescriptor, suggestions: string[], rootPath: string,
similarityLimit: number) { moduleDescriptor: ImportedModuleDescriptor,
suggestions: string[],
similarityLimit: number
) {
// Starting at the specified path, walk the file system to find the // Starting at the specified path, walk the file system to find the
// specified module. // specified module.
let dirPath = rootPath; let dirPath = rootPath;
@ -763,8 +865,7 @@ export class ImportResolver {
// Provide completions only if we're on the last part // Provide completions only if we're on the last part
// of the name. // of the name.
if (i === nameParts.length - 1) { if (i === nameParts.length - 1) {
this._addFilteredSuggestions(dirPath, this._addFilteredSuggestions(dirPath, nameParts[i], suggestions, similarityLimit);
nameParts[i], suggestions, similarityLimit);
} }
dirPath = combinePaths(dirPath, nameParts[i]); dirPath = combinePaths(dirPath, nameParts[i]);
@ -775,9 +876,7 @@ export class ImportResolver {
} }
} }
private _addFilteredSuggestions(dirPath: string, filter: string, suggestions: string[], private _addFilteredSuggestions(dirPath: string, filter: string, suggestions: string[], similarityLimit: number) {
similarityLimit: number) {
const entries = getFileSystemEntries(this.fileSystem, dirPath); const entries = getFileSystemEntries(this.fileSystem, dirPath);
entries.files.forEach(file => { entries.files.forEach(file => {
@ -786,9 +885,10 @@ export class ImportResolver {
if (fileExtension === '.py' || fileExtension === '.pyi' || fileExtension === '.pyd') { if (fileExtension === '.py' || fileExtension === '.pyi' || fileExtension === '.pyd') {
if (fileWithoutExtension !== '__init__') { if (fileWithoutExtension !== '__init__') {
if (!filter || StringUtils.computeCompletionSimilarity( if (
filter, fileWithoutExtension) >= similarityLimit) { !filter ||
StringUtils.computeCompletionSimilarity(filter, fileWithoutExtension) >= similarityLimit
) {
this._addUniqueSuggestion(fileWithoutExtension, suggestions); this._addUniqueSuggestion(fileWithoutExtension, suggestions);
} }
} }

78
server/src/analyzer/importStatementUtils.ts
View File

@ -12,8 +12,15 @@ import { TextEditAction } from '../common/editAction';
import { convertOffsetToPosition } from '../common/positionUtils'; import { convertOffsetToPosition } from '../common/positionUtils';
import { Position } from '../common/textRange'; import { Position } from '../common/textRange';
import { TextRange } from '../common/textRange'; import { TextRange } from '../common/textRange';
import { ImportAsNode, ImportFromAsNode, ImportFromNode, ImportNode, import {
ModuleNameNode, ModuleNode, ParseNodeType } from '../parser/parseNodes'; ImportAsNode,
ImportFromAsNode,
ImportFromNode,
ImportNode,
ModuleNameNode,
ModuleNode,
ParseNodeType
} from '../parser/parseNodes';
import { ParseResults } from '../parser/parser'; import { ParseResults } from '../parser/parser';
import * as AnalyzerNodeInfo from './analyzerNodeInfo'; import * as AnalyzerNodeInfo from './analyzerNodeInfo';
import { ImportResult, ImportType } from './importResult'; import { ImportResult, ImportType } from './importResult';
@ -67,9 +74,11 @@ export function getTopLevelImports(parseTree: ModuleNode): ImportStatements {
return localImports; return localImports;
} }
export function getTextEditsForAutoImportSymbolAddition(symbolName: string, export function getTextEditsForAutoImportSymbolAddition(
importStatement: ImportStatement, parseResults: ParseResults) { symbolName: string,
importStatement: ImportStatement,
parseResults: ParseResults
) {
const textEditList: TextEditAction[] = []; const textEditList: TextEditAction[] = [];
// Scan through the import symbols to find the right insertion point, // Scan through the import symbols to find the right insertion point,
@ -88,15 +97,16 @@ export function getTextEditsForAutoImportSymbolAddition(symbolName: string,
priorImport = curImport; priorImport = curImport;
} }
const insertionOffset = priorImport ? TextRange.getEnd(priorImport) : const insertionOffset = priorImport
(importStatement.node.imports.length > 0 ? ? TextRange.getEnd(priorImport)
importStatement.node.imports[0].start : : importStatement.node.imports.length > 0
importStatement.node.start + importStatement.node.length); ? importStatement.node.imports[0].start
: importStatement.node.start + importStatement.node.length;
const insertionPosition = convertOffsetToPosition(insertionOffset, parseResults.tokenizerOutput.lines); const insertionPosition = convertOffsetToPosition(insertionOffset, parseResults.tokenizerOutput.lines);
textEditList.push({ textEditList.push({
range: { start: insertionPosition, end: insertionPosition }, range: { start: insertionPosition, end: insertionPosition },
replacementText: priorImport ? (', ' + symbolName) : (symbolName + ', ') replacementText: priorImport ? ', ' + symbolName : symbolName + ', '
}); });
} }
} }
@ -104,9 +114,13 @@ export function getTextEditsForAutoImportSymbolAddition(symbolName: string,
return textEditList; return textEditList;
} }
export function getTextEditsForAutoImportInsertion(symbolName: string, importStatements: ImportStatements, export function getTextEditsForAutoImportInsertion(
moduleName: string, importType: ImportType, parseResults: ParseResults): TextEditAction[] { symbolName: string,
importStatements: ImportStatements,
moduleName: string,
importType: ImportType,
parseResults: ParseResults
): TextEditAction[] {
const textEditList: TextEditAction[] = []; const textEditList: TextEditAction[] = [];
// We need to emit a new 'from import' statement. // We need to emit a new 'from import' statement.
@ -124,8 +138,9 @@ export function getTextEditsForAutoImportInsertion(symbolName: string, importSta
// If the import was resolved, use its import type. If it wasn't // If the import was resolved, use its import type. If it wasn't
// resolved, assume that it's the same import type as the previous // resolved, assume that it's the same import type as the previous
// one. // one.
const curImportType: ImportType = curImport.importResult ? const curImportType: ImportType = curImport.importResult
curImport.importResult.importType : prevImportType; ? curImport.importResult.importType
: prevImportType;
if (importType < curImportType) { if (importType < curImportType) {
if (!insertBefore && prevImportType < importType) { if (!insertBefore && prevImportType < importType) {
@ -151,7 +166,6 @@ export function getTextEditsForAutoImportInsertion(symbolName: string, importSta
// If this is the last import, see if we need to create a new group. // If this is the last import, see if we need to create a new group.
if (curImport === importStatements.orderedImports[importStatements.orderedImports.length - 1]) { if (curImport === importStatements.orderedImports[importStatements.orderedImports.length - 1]) {
if (importType > curImportType) { if (importType > curImportType) {
// Add an extra line to create a new group. // Add an extra line to create a new group.
newImportStatement = parseResults.tokenizerOutput.predominantEndOfLineSequence + newImportStatement; newImportStatement = parseResults.tokenizerOutput.predominantEndOfLineSequence + newImportStatement;
@ -178,7 +192,8 @@ export function getTextEditsForAutoImportInsertion(symbolName: string, importSta
insertionPosition = convertOffsetToPosition( insertionPosition = convertOffsetToPosition(
insertBefore ? insertionImport.node.start : TextRange.getEnd(insertionImport.node), insertBefore ? insertionImport.node.start : TextRange.getEnd(insertionImport.node),
parseResults.tokenizerOutput.lines); parseResults.tokenizerOutput.lines
);
} else { } else {
insertionPosition = { line: 0, character: 0 }; insertionPosition = { line: 0, character: 0 };
} }
@ -207,19 +222,20 @@ export function getTextEditsForAutoImportInsertion(symbolName: string, importSta
} }
if (stopHere) { if (stopHere) {
insertionPosition = convertOffsetToPosition(statement.start, insertionPosition = convertOffsetToPosition(statement.start, parseResults.tokenizerOutput.lines);
parseResults.tokenizerOutput.lines);
addNewLineBefore = false; addNewLineBefore = false;
break; break;
} else { } else {
insertionPosition = convertOffsetToPosition( insertionPosition = convertOffsetToPosition(
statement.start + statement.length, statement.start + statement.length,
parseResults.tokenizerOutput.lines); parseResults.tokenizerOutput.lines
);
addNewLineBefore = true; addNewLineBefore = true;
} }
} }
newImportStatement += parseResults.tokenizerOutput.predominantEndOfLineSequence + newImportStatement +=
parseResults.tokenizerOutput.predominantEndOfLineSequence +
parseResults.tokenizerOutput.predominantEndOfLineSequence; parseResults.tokenizerOutput.predominantEndOfLineSequence;
if (addNewLineBefore) { if (addNewLineBefore) {
@ -237,9 +253,7 @@ export function getTextEditsForAutoImportInsertion(symbolName: string, importSta
return textEditList; return textEditList;
} }
function _processImportNode(node: ImportNode, localImports: ImportStatements, function _processImportNode(node: ImportNode, localImports: ImportStatements, followsNonImportStatement: boolean) {
followsNonImportStatement: boolean) {
node.list.forEach(importAsNode => { node.list.forEach(importAsNode => {
const importResult = AnalyzerNodeInfo.getImportInfo(importAsNode.module); const importResult = AnalyzerNodeInfo.getImportInfo(importAsNode.module);
let resolvedPath: string | undefined; let resolvedPath: string | undefined;
@ -271,9 +285,11 @@ function _processImportNode(node: ImportNode, localImports: ImportStatements,
}); });
} }
function _processImportFromNode(node: ImportFromNode, localImports: ImportStatements, function _processImportFromNode(
followsNonImportStatement: boolean) { node: ImportFromNode,
localImports: ImportStatements,
followsNonImportStatement: boolean
) {
const importResult = AnalyzerNodeInfo.getImportInfo(node.module); const importResult = AnalyzerNodeInfo.getImportInfo(node.module);
let resolvedPath: string | undefined; let resolvedPath: string | undefined;
@ -297,9 +313,11 @@ function _processImportFromNode(node: ImportFromNode, localImports: ImportStatem
// Overwrite existing import statements because we always want to prefer // Overwrite existing import statements because we always want to prefer
// 'import from' over 'import'. Also, overwrite existing 'import from' if // 'import from' over 'import'. Also, overwrite existing 'import from' if
// the module name is shorter. // the module name is shorter.
if (!prevEntry || prevEntry.node.nodeType === ParseNodeType.Import || if (
prevEntry.moduleName.length > localImport.moduleName.length) { !prevEntry ||
prevEntry.node.nodeType === ParseNodeType.Import ||
prevEntry.moduleName.length > localImport.moduleName.length
) {
localImports.mapByFilePath.set(resolvedPath, localImport); localImports.mapByFilePath.set(resolvedPath, localImport);
} }
} }

205
server/src/analyzer/parseTreeUtils.ts
View File

@ -12,9 +12,23 @@ import { convertPositionToOffset } from '../common/positionUtils';
import { Position } from '../common/textRange'; import { Position } from '../common/textRange';
import { TextRange } from '../common/textRange'; import { TextRange } from '../common/textRange';
import { TextRangeCollection } from '../common/textRangeCollection'; import { TextRangeCollection } from '../common/textRangeCollection';
import { ArgumentCategory, AssignmentExpressionNode, ClassNode, EvaluationScopeNode, import {
ExecutionScopeNode, ExpressionNode, FunctionNode, isExpressionNode, LambdaNode, ModuleNode, ArgumentCategory,
ParameterCategory, ParseNode, ParseNodeType, StatementNode, SuiteNode } from '../parser/parseNodes'; AssignmentExpressionNode,
ClassNode,
EvaluationScopeNode,
ExecutionScopeNode,
ExpressionNode,
FunctionNode,
isExpressionNode,
LambdaNode,
ModuleNode,
ParameterCategory,
ParseNode,
ParseNodeType,
StatementNode,
SuiteNode
} from '../parser/parseNodes';
import { KeywordType, OperatorType, StringTokenFlags } from '../parser/tokenizerTypes'; import { KeywordType, OperatorType, StringTokenFlags } from '../parser/tokenizerTypes';
import { decodeDocString } from './docStringUtils'; import { decodeDocString } from './docStringUtils';
import { ParseTreeWalker } from './parseTreeWalker'; import { ParseTreeWalker } from './parseTreeWalker';
@ -39,9 +53,11 @@ export function getNodeDepth(node: ParseNode): number {
} }
// Returns the deepest node that contains the specified position. // Returns the deepest node that contains the specified position.
export function findNodeByPosition(node: ParseNode, position: Position, export function findNodeByPosition(
lines: TextRangeCollection<TextRange>): ParseNode | undefined { node: ParseNode,
position: Position,
lines: TextRangeCollection<TextRange>
): ParseNode | undefined {
const offset = convertPositionToOffset(position, lines); const offset = convertPositionToOffset(position, lines);
if (offset === undefined) { if (offset === undefined) {
return undefined; return undefined;
@ -80,13 +96,15 @@ export function printExpression(node: ExpressionNode, flags = PrintExpressionFla
} }
case ParseNodeType.MemberAccess: { case ParseNodeType.MemberAccess: {
return printExpression(node.leftExpression, flags) + '.' + return printExpression(node.leftExpression, flags) + '.' + node.memberName.value;
node.memberName.value;
} }
case ParseNodeType.Call: { case ParseNodeType.Call: {
return printExpression(node.leftExpression, flags) + '(' + return (
node.arguments.map(arg => { printExpression(node.leftExpression, flags) +
'(' +
node.arguments
.map(arg => {
let argStr = ''; let argStr = '';
if (arg.argumentCategory === ArgumentCategory.UnpackedList) { if (arg.argumentCategory === ArgumentCategory.UnpackedList) {
argStr = '*'; argStr = '*';
@ -98,25 +116,33 @@ export function printExpression(node: ExpressionNode, flags = PrintExpressionFla
} }
argStr += printExpression(arg.valueExpression, flags); argStr += printExpression(arg.valueExpression, flags);
return argStr; return argStr;
}).join(', ') + })
')'; .join(', ') +
')'
);
} }
case ParseNodeType.Index: { case ParseNodeType.Index: {
return printExpression(node.baseExpression, flags) + '[' + return (
printExpression(node.baseExpression, flags) +
'[' +
node.items.items.map(item => printExpression(item, flags)).join(', ') + node.items.items.map(item => printExpression(item, flags)).join(', ') +
']'; ']'
);
} }
case ParseNodeType.UnaryOperation: { case ParseNodeType.UnaryOperation: {
return printOperator(node.operator) + ' ' + return printOperator(node.operator) + ' ' + printExpression(node.expression, flags);
printExpression(node.expression, flags);
} }
case ParseNodeType.BinaryOperation: { case ParseNodeType.BinaryOperation: {
return printExpression(node.leftExpression, flags) + ' ' + return (
printOperator(node.operator) + ' ' + printExpression(node.leftExpression, flags) +
printExpression(node.rightExpression, flags); ' ' +
printOperator(node.operator) +
' ' +
printExpression(node.rightExpression, flags)
);
} }
case ParseNodeType.Number: { case ParseNodeType.Number: {
@ -128,12 +154,14 @@ export function printExpression(node: ExpressionNode, flags = PrintExpressionFla
} }
case ParseNodeType.StringList: { case ParseNodeType.StringList: {
if ((flags & PrintExpressionFlags.ForwardDeclarations) && node.typeAnnotation) { if (flags & PrintExpressionFlags.ForwardDeclarations && node.typeAnnotation) {
return printExpression(node.typeAnnotation, flags); return printExpression(node.typeAnnotation, flags);
} else { } else {
return node.strings.map(str => { return node.strings
.map(str => {
return printExpression(str, flags); return printExpression(str, flags);
}).join(' '); })
.join(' ');
} }
} }
@ -173,24 +201,25 @@ export function printExpression(node: ExpressionNode, flags = PrintExpressionFla
} }
case ParseNodeType.Assignment: { case ParseNodeType.Assignment: {
return printExpression(node.leftExpression, flags) + ' = ' + return printExpression(node.leftExpression, flags) + ' = ' + printExpression(node.rightExpression, flags);
printExpression(node.rightExpression, flags);
} }
case ParseNodeType.AssignmentExpression: { case ParseNodeType.AssignmentExpression: {
return printExpression(node.name, flags) + ' := ' + return printExpression(node.name, flags) + ' := ' + printExpression(node.rightExpression, flags);
printExpression(node.rightExpression, flags);
} }
case ParseNodeType.TypeAnnotation: { case ParseNodeType.TypeAnnotation: {
return printExpression(node.valueExpression, flags) + ': ' + return printExpression(node.valueExpression, flags) + ': ' + printExpression(node.typeAnnotation, flags);
printExpression(node.typeAnnotation, flags);
} }
case ParseNodeType.AugmentedAssignment: { case ParseNodeType.AugmentedAssignment: {
return printExpression(node.leftExpression, flags) + ' ' + return (
printOperator(node.operator) + ' ' + printExpression(node.leftExpression, flags) +
printExpression(node.rightExpression, flags); ' ' +
printOperator(node.operator) +
' ' +
printExpression(node.rightExpression, flags)
);
} }
case ParseNodeType.Await: { case ParseNodeType.Await: {
@ -198,9 +227,13 @@ export function printExpression(node: ExpressionNode, flags = PrintExpressionFla
} }
case ParseNodeType.Ternary: { case ParseNodeType.Ternary: {
return printExpression(node.ifExpression, flags) + ' if ' + return (
printExpression(node.testExpression, flags) + ' else ' + printExpression(node.ifExpression, flags) +
printExpression(node.elseExpression, flags); ' if ' +
printExpression(node.testExpression, flags) +
' else ' +
printExpression(node.elseExpression, flags)
);
} }
case ParseNodeType.List: { case ParseNodeType.List: {
@ -251,16 +284,23 @@ export function printExpression(node: ExpressionNode, flags = PrintExpressionFla
listStr = `${keyStr}: ${valueStr}`; listStr = `${keyStr}: ${valueStr}`;
} }
return listStr + ' ' + return (
node.comprehensions.map(expr => { listStr +
' ' +
node.comprehensions
.map(expr => {
if (expr.nodeType === ParseNodeType.ListComprehensionFor) { if (expr.nodeType === ParseNodeType.ListComprehensionFor) {
return `${ expr.isAsync ? 'async ' : '' }for ` + return (
`${expr.isAsync ? 'async ' : ''}for ` +
printExpression(expr.targetExpression, flags) + printExpression(expr.targetExpression, flags) +
` in ${ printExpression(expr.iterableExpression, flags) }`; ` in ${printExpression(expr.iterableExpression, flags)}`
);
} else { } else {
return `if ${printExpression(expr.testExpression, flags)}`; return `if ${printExpression(expr.testExpression, flags)}`;
} }
}).join(' '); })
.join(' ')
);
} }
case ParseNodeType.Slice: { case ParseNodeType.Slice: {
@ -278,7 +318,10 @@ export function printExpression(node: ExpressionNode, flags = PrintExpressionFla
} }
case ParseNodeType.Lambda: { case ParseNodeType.Lambda: {
return 'lambda ' + node.parameters.map(param => { return (
'lambda ' +
node.parameters
.map(param => {
let paramStr = ''; let paramStr = '';
if (param.category === ParameterCategory.VarArgList) { if (param.category === ParameterCategory.VarArgList) {
@ -295,7 +338,11 @@ export function printExpression(node: ExpressionNode, flags = PrintExpressionFla
paramStr += ' = ' + printExpression(param.defaultValue, flags); paramStr += ' = ' + printExpression(param.defaultValue, flags);
} }
return paramStr; return paramStr;
}).join(', ') + ': ' + printExpression(node.expression, flags); })
.join(', ') +
': ' +
printExpression(node.expression, flags)
);
} }
case ParseNodeType.Constant: { case ParseNodeType.Constant: {
@ -314,8 +361,10 @@ export function printExpression(node: ExpressionNode, flags = PrintExpressionFla
case ParseNodeType.Dictionary: { case ParseNodeType.Dictionary: {
return `{ ${node.entries.map(entry => { return `{ ${node.entries.map(entry => {
if (entry.nodeType === ParseNodeType.DictionaryKeyEntry) { if (entry.nodeType === ParseNodeType.DictionaryKeyEntry) {
return `${ printExpression(entry.keyExpression, flags) }: ` + return (
`${ printExpression(entry.valueExpression, flags) }`; `${printExpression(entry.keyExpression, flags)}: ` +
`${printExpression(entry.valueExpression, flags)}`
);
} else { } else {
return printExpression(entry, flags); return printExpression(entry, flags);
} }
@ -423,9 +472,7 @@ export function getEnclosingModule(node: ParseNode): ModuleNode {
return undefined!; return undefined!;
} }
export function getEnclosingClassOrModule(node: ParseNode, export function getEnclosingClassOrModule(node: ParseNode, stopAtFunction = false): ClassNode | ModuleNode | undefined {
stopAtFunction = false): ClassNode | ModuleNode | undefined {
let curNode = node.parent; let curNode = node.parent;
while (curNode) { while (curNode) {
if (curNode.nodeType === ParseNodeType.Class) { if (curNode.nodeType === ParseNodeType.Class) {
@ -465,9 +512,9 @@ export function getEnclosingFunction(node: ParseNode): FunctionNode | undefined
return undefined; return undefined;
} }
export function getEvaluationNodeForAssignmentExpression(node: AssignmentExpressionNode): export function getEvaluationNodeForAssignmentExpression(
LambdaNode | FunctionNode | ModuleNode | undefined { node: AssignmentExpressionNode
): LambdaNode | FunctionNode | ModuleNode | undefined {
// PEP 572 indicates that the evaluation node for an assignment expression // PEP 572 indicates that the evaluation node for an assignment expression
// target is the containing lambda, function or module, but not a class. // target is the containing lambda, function or module, but not a class.
let curNode: ParseNode | undefined = getEvaluationScopeNode(node); let curNode: ParseNode | undefined = getEvaluationScopeNode(node);
@ -547,9 +594,10 @@ export function getExecutionScopeNode(node: ParseNode): ExecutionScopeNode {
// Classes are not considered execution scope because they are executed // Classes are not considered execution scope because they are executed
// within the context of their containing module or function. Likewise, list // within the context of their containing module or function. Likewise, list
// comprehensions are executed within their container. // comprehensions are executed within their container.
while (evaluationScope.nodeType === ParseNodeType.Class || while (
evaluationScope.nodeType === ParseNodeType.ListComprehension) { evaluationScope.nodeType === ParseNodeType.Class ||
evaluationScope.nodeType === ParseNodeType.ListComprehension
) {
evaluationScope = getEvaluationScopeNode(evaluationScope.parent!); evaluationScope = getEvaluationScopeNode(evaluationScope.parent!);
} }
@ -592,9 +640,14 @@ export function isSuiteEmpty(node: SuiteNode): boolean {
export function isMatchingExpression(expression1: ExpressionNode, expression2: ExpressionNode): boolean { export function isMatchingExpression(expression1: ExpressionNode, expression2: ExpressionNode): boolean {
if (expression1.nodeType === ParseNodeType.Name && expression2.nodeType === ParseNodeType.Name) { if (expression1.nodeType === ParseNodeType.Name && expression2.nodeType === ParseNodeType.Name) {
return expression1.value === expression2.value; return expression1.value === expression2.value;
} else if (expression1.nodeType === ParseNodeType.MemberAccess && expression2.nodeType === ParseNodeType.MemberAccess) { } else if (
return isMatchingExpression(expression1.leftExpression, expression2.leftExpression) && expression1.nodeType === ParseNodeType.MemberAccess &&
expression1.memberName.value === expression2.memberName.value; expression2.nodeType === ParseNodeType.MemberAccess
) {
return (
isMatchingExpression(expression1.leftExpression, expression2.leftExpression) &&
expression1.memberName.value === expression2.memberName.value
);
} }
return false; return false;
@ -609,10 +662,12 @@ export function isWithinDefaultParamInitializer(node: ParseNode) {
return true; return true;
} }
if (curNode.nodeType === ParseNodeType.Lambda || if (
curNode.nodeType === ParseNodeType.Lambda ||
curNode.nodeType === ParseNodeType.Function || curNode.nodeType === ParseNodeType.Function ||
curNode.nodeType === ParseNodeType.Class || curNode.nodeType === ParseNodeType.Class ||
curNode.nodeType === ParseNodeType.Module) { curNode.nodeType === ParseNodeType.Module
) {
return false; return false;
} }
@ -636,8 +691,7 @@ export function getDocString(statements: StatementNode[]): string | undefined {
// If the first statement in the suite isn't a StringNode, // If the first statement in the suite isn't a StringNode,
// assume there is no docString. // assume there is no docString.
const statementList = statements[0]; const statementList = statements[0];
if (statementList.statements.length === 0 || if (statementList.statements.length === 0 || statementList.statements[0].nodeType !== ParseNodeType.StringList) {
statementList.statements[0].nodeType !== ParseNodeType.StringList) {
return undefined; return undefined;
} }
@ -658,22 +712,27 @@ export function getDocString(statements: StatementNode[]): string | undefined {
// This pattern is commonly used to set the default values that are // This pattern is commonly used to set the default values that are
// not specified in the original list. // not specified in the original list.
export function isAssignmentToDefaultsFollowingNamedTuple(callNode: ParseNode): boolean { export function isAssignmentToDefaultsFollowingNamedTuple(callNode: ParseNode): boolean {
if (callNode.nodeType !== ParseNodeType.Call || !callNode.parent || if (
callNode.nodeType !== ParseNodeType.Call ||
!callNode.parent ||
callNode.parent.nodeType !== ParseNodeType.Assignment || callNode.parent.nodeType !== ParseNodeType.Assignment ||
callNode.parent.leftExpression.nodeType !== ParseNodeType.Name || callNode.parent.leftExpression.nodeType !== ParseNodeType.Name ||
!callNode.parent.parent || !callNode.parent.parent ||
callNode.parent.parent.nodeType !== ParseNodeType.StatementList) { callNode.parent.parent.nodeType !== ParseNodeType.StatementList
) {
return false; return false;
} }
const namedTupleAssignedName = callNode.parent.leftExpression.value; const namedTupleAssignedName = callNode.parent.leftExpression.value;
const statementList = callNode.parent.parent; const statementList = callNode.parent.parent;
if (statementList.statements[0] !== callNode.parent || if (
statementList.statements[0] !== callNode.parent ||
!statementList.parent || !statementList.parent ||
!(statementList.parent.nodeType === ParseNodeType.Module || !(
statementList.parent.nodeType === ParseNodeType.Suite)) { statementList.parent.nodeType === ParseNodeType.Module ||
statementList.parent.nodeType === ParseNodeType.Suite
)
) {
return false; return false;
} }
@ -699,15 +758,17 @@ export function isAssignmentToDefaultsFollowingNamedTuple(callNode: ParseNode):
if (nextStatement.statements[0].nodeType === ParseNodeType.Assignment) { if (nextStatement.statements[0].nodeType === ParseNodeType.Assignment) {
const assignNode = nextStatement.statements[0]; const assignNode = nextStatement.statements[0];
if (assignNode.leftExpression.nodeType === ParseNodeType.MemberAccess && if (
assignNode.leftExpression.memberName.value === '__defaults__') { assignNode.leftExpression.nodeType === ParseNodeType.MemberAccess &&
assignNode.leftExpression.memberName.value === '__defaults__'
) {
const defaultTarget = assignNode.leftExpression.leftExpression; const defaultTarget = assignNode.leftExpression.leftExpression;
if (defaultTarget.nodeType === ParseNodeType.MemberAccess && if (
defaultTarget.nodeType === ParseNodeType.MemberAccess &&
defaultTarget.memberName.value === '__new__' && defaultTarget.memberName.value === '__new__' &&
defaultTarget.leftExpression.nodeType === ParseNodeType.Name && defaultTarget.leftExpression.nodeType === ParseNodeType.Name &&
defaultTarget.leftExpression.value === namedTupleAssignedName) { defaultTarget.leftExpression.value === namedTupleAssignedName
) {
return true; return true;
} }
} }

84
server/src/analyzer/parseTreeWalker.ts
View File

@ -8,19 +8,74 @@
*/ */
import { fail } from '../common/debug'; import { fail } from '../common/debug';
import { ArgumentNode, AssertNode, AssignmentExpressionNode, AssignmentNode, import {
AugmentedAssignmentNode, AwaitNode, BinaryOperationNode, BreakNode, CallNode, ArgumentNode,
ClassNode, ConstantNode, ContinueNode, DecoratorNode, DelNode, AssertNode,
DictionaryExpandEntryNode, DictionaryKeyEntryNode, DictionaryNode, EllipsisNode, AssignmentExpressionNode,
ErrorNode, ExceptNode, FormatStringNode, ForNode, FunctionNode, GlobalNode, IfNode, AssignmentNode,
ImportAsNode, ImportFromAsNode, ImportFromNode, ImportNode, IndexItemsNode, AugmentedAssignmentNode,
IndexNode, LambdaNode, ListComprehensionForNode, ListComprehensionIfNode, AwaitNode,
ListComprehensionNode, ListNode, MemberAccessNode, ModuleNameNode, ModuleNode, NameNode, BinaryOperationNode,
NonlocalNode, NumberNode, ParameterNode, ParseNode, ParseNodeArray, ParseNodeType, BreakNode,
PassNode, RaiseNode, ReturnNode, SetNode, SliceNode, StatementListNode, CallNode,
StringListNode, StringNode, SuiteNode, TernaryNode, TryNode, ClassNode,
TupleNode, TypeAnnotationNode, UnaryOperationNode, UnpackNode, ConstantNode,
WhileNode, WithItemNode, WithNode, YieldFromNode, YieldNode } from '../parser/parseNodes'; ContinueNode,
DecoratorNode,
DelNode,
DictionaryExpandEntryNode,
DictionaryKeyEntryNode,
DictionaryNode,
EllipsisNode,
ErrorNode,
ExceptNode,
FormatStringNode,
ForNode,
FunctionNode,
GlobalNode,
IfNode,
ImportAsNode,
ImportFromAsNode,
ImportFromNode,
ImportNode,
IndexItemsNode,
IndexNode,
LambdaNode,
ListComprehensionForNode,
ListComprehensionIfNode,
ListComprehensionNode,
ListNode,
MemberAccessNode,
ModuleNameNode,
ModuleNode,
NameNode,
NonlocalNode,
NumberNode,
ParameterNode,
ParseNode,
ParseNodeArray,
ParseNodeType,
PassNode,
RaiseNode,
ReturnNode,
SetNode,
SliceNode,
StatementListNode,
StringListNode,
StringNode,
SuiteNode,
TernaryNode,
TryNode,
TupleNode,
TypeAnnotationNode,
UnaryOperationNode,
UnpackNode,
WhileNode,
WithItemNode,
WithNode,
YieldFromNode,
YieldNode
} from '../parser/parseNodes';
// To use this class, create a subclass and override the // To use this class, create a subclass and override the
// visitXXX methods that you want to handle. // visitXXX methods that you want to handle.
@ -229,8 +284,7 @@ export class ParseTreeWalker {
case ParseNodeType.Function: case ParseNodeType.Function:
if (this.visitFunction(node)) { if (this.visitFunction(node)) {
return [...node.decorators, node.name, ...node.parameters, return [...node.decorators, node.name, ...node.parameters, node.returnTypeAnnotation, node.suite];
node.returnTypeAnnotation, node.suite];
} }
break; break;

213
server/src/analyzer/program.ts
View File

@ -17,8 +17,12 @@ import { Diagnostic } from '../common/diagnostic';
import { FileDiagnostics } from '../common/diagnosticSink'; import { FileDiagnostics } from '../common/diagnosticSink';
import { FileEditAction, TextEditAction } from '../common/editAction'; import { FileEditAction, TextEditAction } from '../common/editAction';
import { import {
combinePaths, getDirectoryPath, getRelativePath, makeDirectories, combinePaths,
normalizePath, stripFileExtension getDirectoryPath,
getRelativePath,
makeDirectories,
normalizePath,
stripFileExtension
} from '../common/pathUtils'; } from '../common/pathUtils';
import { DocumentRange, doRangesOverlap, Position, Range } from '../common/textRange'; import { DocumentRange, doRangesOverlap, Position, Range } from '../common/textRange';
import { Duration, timingStats } from '../common/timing'; import { Duration, timingStats } from '../common/timing';
@ -82,8 +86,11 @@ export class Program {
private _configOptions: ConfigOptions; private _configOptions: ConfigOptions;
private _importResolver: ImportResolver; private _importResolver: ImportResolver;
constructor(initialImportResolver: ImportResolver, initialConfigOptions: ConfigOptions, constructor(
console?: ConsoleInterface) { initialImportResolver: ImportResolver,
initialConfigOptions: ConfigOptions,
console?: ConsoleInterface
) {
this._console = console || new StandardConsole(); this._console = console || new StandardConsole();
this._evaluator = createTypeEvaluator(this._lookUpImport); this._evaluator = createTypeEvaluator(this._lookUpImport);
this._importResolver = initialImportResolver; this._importResolver = initialImportResolver;
@ -140,10 +147,11 @@ export class Program {
let sourceFileCount = 0; let sourceFileCount = 0;
this._sourceFileList.forEach(fileInfo => { this._sourceFileList.forEach(fileInfo => {
if (fileInfo.sourceFile.isParseRequired() || if (
fileInfo.sourceFile.isParseRequired() ||
fileInfo.sourceFile.isBindingRequired() || fileInfo.sourceFile.isBindingRequired() ||
fileInfo.sourceFile.isCheckingRequired()) { fileInfo.sourceFile.isCheckingRequired()
) {
if ((!this._configOptions.checkOnlyOpenFiles && fileInfo.isTracked) || fileInfo.isOpenByClient) { if ((!this._configOptions.checkOnlyOpenFiles && fileInfo.isTracked) || fileInfo.isOpenByClient) {
sourceFileCount++; sourceFileCount++;
} }
@ -272,13 +280,10 @@ export class Program {
analyze(maxTime?: MaxAnalysisTime, interactiveMode?: boolean): boolean { analyze(maxTime?: MaxAnalysisTime, interactiveMode?: boolean): boolean {
const elapsedTime = new Duration(); const elapsedTime = new Duration();
const openFiles = this._sourceFileList.filter( const openFiles = this._sourceFileList.filter(sf => sf.isOpenByClient && sf.sourceFile.isCheckingRequired());
sf => sf.isOpenByClient && sf.sourceFile.isCheckingRequired()
);
if (openFiles.length > 0) { if (openFiles.length > 0) {
const effectiveMaxTime = maxTime ? const effectiveMaxTime = maxTime ? maxTime.openFilesTimeInMs : Number.MAX_VALUE;
maxTime.openFilesTimeInMs : Number.MAX_VALUE;
// Check the open files. // Check the open files.
for (const sourceFileInfo of openFiles) { for (const sourceFileInfo of openFiles) {
@ -300,9 +305,11 @@ export class Program {
if (!this._configOptions.checkOnlyOpenFiles) { if (!this._configOptions.checkOnlyOpenFiles) {
// Do type analysis of remaining files. // Do type analysis of remaining files.
const allFiles = this._sourceFileList; const allFiles = this._sourceFileList;
const effectiveMaxTime = maxTime ? const effectiveMaxTime = maxTime
(interactiveMode ? maxTime.openFilesTimeInMs : maxTime.noOpenFilesTimeInMs) : ? interactiveMode
Number.MAX_VALUE; ? maxTime.openFilesTimeInMs
: maxTime.noOpenFilesTimeInMs
: Number.MAX_VALUE;
// Now do type parsing and analysis of the remaining. // Now do type parsing and analysis of the remaining.
for (const sourceFileInfo of allFiles) { for (const sourceFileInfo of allFiles) {
@ -320,8 +327,10 @@ export class Program {
// Prints import dependency information for each of the files in // Prints import dependency information for each of the files in
// the program, skipping any typeshed files. // the program, skipping any typeshed files.
printDependencies(projectRootDir: string, verbose: boolean) { printDependencies(projectRootDir: string, verbose: boolean) {
const sortedFiles = this._sourceFileList.filter(s => !s.isTypeshedFile).sort((a, b) => { const sortedFiles = this._sourceFileList
return (a.sourceFile.getFilePath() < b.sourceFile.getFilePath()) ? 1 : -1; .filter(s => !s.isTypeshedFile)
.sort((a, b) => {
return a.sourceFile.getFilePath() < b.sourceFile.getFilePath() ? 1 : -1;
}); });
const zeroImportFiles: SourceFile[] = []; const zeroImportFiles: SourceFile[] = [];
@ -336,16 +345,18 @@ export class Program {
this._console.log(`${filePath}`); this._console.log(`${filePath}`);
this._console.log(` Imports ${ sfInfo.imports.length } ` + this._console.log(
`file${ sfInfo.imports.length === 1 ? '' : 's' }`); ` Imports ${sfInfo.imports.length} ` + `file${sfInfo.imports.length === 1 ? '' : 's'}`
);
if (verbose) { if (verbose) {
sfInfo.imports.forEach(importInfo => { sfInfo.imports.forEach(importInfo => {
this._console.log(` ${importInfo.sourceFile.getFilePath()}`); this._console.log(` ${importInfo.sourceFile.getFilePath()}`);
}); });
} }
this._console.log(` Imported by ${ sfInfo.importedBy.length } ` + this._console.log(
`file${ sfInfo.importedBy.length === 1 ? '' : 's' }`); ` Imported by ${sfInfo.importedBy.length} ` + `file${sfInfo.importedBy.length === 1 ? '' : 's'}`
);
if (verbose) { if (verbose) {
sfInfo.importedBy.forEach(importInfo => { sfInfo.importedBy.forEach(importInfo => {
this._console.log(` ${importInfo.sourceFile.getFilePath()}`); this._console.log(` ${importInfo.sourceFile.getFilePath()}`);
@ -359,8 +370,9 @@ export class Program {
if (zeroImportFiles.length > 0) { if (zeroImportFiles.length > 0) {
this._console.log(''); this._console.log('');
this._console.log(`${ zeroImportFiles.length } file${ zeroImportFiles.length === 1 ? '' : 's' }` + this._console.log(
` not explicitly imported`); `${zeroImportFiles.length} file${zeroImportFiles.length === 1 ? '' : 's'}` + ` not explicitly imported`
);
zeroImportFiles.forEach(importFile => { zeroImportFiles.forEach(importFile => {
this._console.log(` ${importFile.getFilePath()}`); this._console.log(` ${importFile.getFilePath()}`);
}); });
@ -482,7 +494,7 @@ export class Program {
symbolTable, symbolTable,
docString docString
}; };
} };
// Build a map of all modules within this program and the module- // Build a map of all modules within this program and the module-
// level scope that contains the symbol table for the module. // level scope that contains the symbol table for the module.
@ -553,9 +565,11 @@ export class Program {
// it imports (recursively) and ensures that all such files. If any of these files // it imports (recursively) and ensures that all such files. If any of these files
// have already been checked (they and their recursive imports have completed the // have already been checked (they and their recursive imports have completed the
// check phase), they are not included in the results. // check phase), they are not included in the results.
private _getImportsRecursive(file: SourceFileInfo, closureMap: Map<string, SourceFileInfo>, private _getImportsRecursive(
recursionCount: number) { file: SourceFileInfo,
closureMap: Map<string, SourceFileInfo>,
recursionCount: number
) {
// If the file is already in the closure map, we found a cyclical // If the file is already in the closure map, we found a cyclical
// dependency. Don't recur further. // dependency. Don't recur further.
const filePath = file.sourceFile.getFilePath(); const filePath = file.sourceFile.getFilePath();
@ -579,10 +593,11 @@ export class Program {
} }
} }
private _detectAndReportImportCycles(sourceFileInfo: SourceFileInfo, private _detectAndReportImportCycles(
sourceFileInfo: SourceFileInfo,
dependencyChain: SourceFileInfo[] = [], dependencyChain: SourceFileInfo[] = [],
dependencyMap = new Map<string, boolean>()): void { dependencyMap = new Map<string, boolean>()
): void {
// Don't bother checking for typestub files or third-party files. // Don't bother checking for typestub files or third-party files.
if (sourceFileInfo.sourceFile.isStubFile() || sourceFileInfo.isThirdPartyImport) { if (sourceFileInfo.sourceFile.isStubFile() || sourceFileInfo.isThirdPartyImport) {
return; return;
@ -634,9 +649,7 @@ export class Program {
firstSourceFile.sourceFile.addCircularDependency(circDep); firstSourceFile.sourceFile.addCircularDependency(circDep);
} }
private _markFileDirtyRecursive(sourceFileInfo: SourceFileInfo, private _markFileDirtyRecursive(sourceFileInfo: SourceFileInfo, markMap: Map<string, boolean>) {
markMap: Map<string, boolean>) {
const filePath = sourceFileInfo.sourceFile.getFilePath(); const filePath = sourceFileInfo.sourceFile.getFilePath();
// Don't mark it again if it's already been visited. // Don't mark it again if it's already been visited.
@ -656,7 +669,9 @@ export class Program {
this._sourceFileList.forEach(sourceFileInfo => { this._sourceFileList.forEach(sourceFileInfo => {
if ((!options.checkOnlyOpenFiles && sourceFileInfo.isTracked) || sourceFileInfo.isOpenByClient) { if ((!options.checkOnlyOpenFiles && sourceFileInfo.isTracked) || sourceFileInfo.isOpenByClient) {
const diagnostics = sourceFileInfo.sourceFile.getDiagnostics( const diagnostics = sourceFileInfo.sourceFile.getDiagnostics(
options, sourceFileInfo.diagnosticsVersion); options,
sourceFileInfo.diagnosticsVersion
);
if (diagnostics !== undefined) { if (diagnostics !== undefined) {
fileDiagnostics.push({ fileDiagnostics.push({
filePath: sourceFileInfo.sourceFile.getFilePath(), filePath: sourceFileInfo.sourceFile.getFilePath(),
@ -665,8 +680,7 @@ export class Program {
// Update the cached diagnosticsVersion so we can determine // Update the cached diagnosticsVersion so we can determine
// whether there are any updates next time we call getDiagnostics. // whether there are any updates next time we call getDiagnostics.
sourceFileInfo.diagnosticsVersion = sourceFileInfo.diagnosticsVersion = sourceFileInfo.sourceFile.getDiagnosticVersion();
sourceFileInfo.sourceFile.getDiagnosticVersion();
} }
} }
}); });
@ -690,9 +704,7 @@ export class Program {
}); });
} }
getDefinitionsForPosition(filePath: string, position: Position): getDefinitionsForPosition(filePath: string, position: Position): DocumentRange[] | undefined {
DocumentRange[] | undefined {
const sourceFileInfo = this._sourceFileMap.get(filePath); const sourceFileInfo = this._sourceFileMap.get(filePath);
if (!sourceFileInfo) { if (!sourceFileInfo) {
return undefined; return undefined;
@ -703,9 +715,11 @@ export class Program {
return sourceFileInfo.sourceFile.getDefinitionsForPosition(position, this._evaluator); return sourceFileInfo.sourceFile.getDefinitionsForPosition(position, this._evaluator);
} }
getReferencesForPosition(filePath: string, position: Position, getReferencesForPosition(
includeDeclaration: boolean): DocumentRange[] | undefined { filePath: string,
position: Position,
includeDeclaration: boolean
): DocumentRange[] | undefined {
const sourceFileInfo = this._sourceFileMap.get(filePath); const sourceFileInfo = this._sourceFileMap.get(filePath);
if (!sourceFileInfo) { if (!sourceFileInfo) {
return undefined; return undefined;
@ -714,7 +728,10 @@ export class Program {
this._bindFile(sourceFileInfo); this._bindFile(sourceFileInfo);
const referencesResult = sourceFileInfo.sourceFile.getReferencesForPosition( const referencesResult = sourceFileInfo.sourceFile.getReferencesForPosition(
position, includeDeclaration, this._evaluator); position,
includeDeclaration,
this._evaluator
);
if (!referencesResult) { if (!referencesResult) {
return undefined; return undefined;
@ -726,8 +743,7 @@ export class Program {
if (curSourceFileInfo !== sourceFileInfo) { if (curSourceFileInfo !== sourceFileInfo) {
this._bindFile(curSourceFileInfo); this._bindFile(curSourceFileInfo);
curSourceFileInfo.sourceFile.addReferences(referencesResult, curSourceFileInfo.sourceFile.addReferences(referencesResult, includeDeclaration, this._evaluator);
includeDeclaration, this._evaluator);
} }
} }
} }
@ -740,8 +756,7 @@ export class Program {
if (sourceFileInfo) { if (sourceFileInfo) {
this._bindFile(sourceFileInfo); this._bindFile(sourceFileInfo);
sourceFileInfo.sourceFile.addHierarchicalSymbolsForDocument( sourceFileInfo.sourceFile.addHierarchicalSymbolsForDocument(symbolList, this._evaluator);
symbolList, this._evaluator);
} }
} }
@ -755,14 +770,11 @@ export class Program {
for (const sourceFileInfo of this._sourceFileList) { for (const sourceFileInfo of this._sourceFileList) {
this._bindFile(sourceFileInfo); this._bindFile(sourceFileInfo);
sourceFileInfo.sourceFile.addSymbolsForDocument( sourceFileInfo.sourceFile.addSymbolsForDocument(symbolList, this._evaluator, query);
symbolList, this._evaluator, query);
} }
} }
getHoverForPosition(filePath: string, position: Position): getHoverForPosition(filePath: string, position: Position): HoverResults | undefined {
HoverResults | undefined {
const sourceFileInfo = this._sourceFileMap.get(filePath); const sourceFileInfo = this._sourceFileMap.get(filePath);
if (!sourceFileInfo) { if (!sourceFileInfo) {
return undefined; return undefined;
@ -773,9 +785,7 @@ export class Program {
return sourceFileInfo.sourceFile.getHoverForPosition(position, this._evaluator); return sourceFileInfo.sourceFile.getHoverForPosition(position, this._evaluator);
} }
getSignatureHelpForPosition(filePath: string, position: Position): getSignatureHelpForPosition(filePath: string, position: Position): SignatureHelpResults | undefined {
SignatureHelpResults | undefined {
const sourceFileInfo = this._sourceFileMap.get(filePath); const sourceFileInfo = this._sourceFileMap.get(filePath);
if (!sourceFileInfo) { if (!sourceFileInfo) {
return undefined; return undefined;
@ -783,13 +793,10 @@ export class Program {
this._bindFile(sourceFileInfo); this._bindFile(sourceFileInfo);
return sourceFileInfo.sourceFile.getSignatureHelpForPosition( return sourceFileInfo.sourceFile.getSignatureHelpForPosition(position, this._lookUpImport, this._evaluator);
position, this._lookUpImport, this._evaluator);
} }
getCompletionsForPosition(filePath: string, position: Position, getCompletionsForPosition(filePath: string, position: Position, workspacePath: string): CompletionList | undefined {
workspacePath: string): CompletionList | undefined {
const sourceFileInfo = this._sourceFileMap.get(filePath); const sourceFileInfo = this._sourceFileMap.get(filePath);
if (!sourceFileInfo) { if (!sourceFileInfo) {
return undefined; return undefined;
@ -798,9 +805,14 @@ export class Program {
this._bindFile(sourceFileInfo); this._bindFile(sourceFileInfo);
return sourceFileInfo.sourceFile.getCompletionsForPosition( return sourceFileInfo.sourceFile.getCompletionsForPosition(
position, workspacePath, this._configOptions, position,
this._importResolver, this._lookUpImport, this._evaluator, workspacePath,
() => this._buildModuleSymbolsMap(sourceFileInfo)); this._configOptions,
this._importResolver,
this._lookUpImport,
this._evaluator,
() => this._buildModuleSymbolsMap(sourceFileInfo)
);
} }
resolveCompletionItem(filePath: string, completionItem: CompletionItem) { resolveCompletionItem(filePath: string, completionItem: CompletionItem) {
@ -812,13 +824,16 @@ export class Program {
this._bindFile(sourceFileInfo); this._bindFile(sourceFileInfo);
sourceFileInfo.sourceFile.resolveCompletionItem( sourceFileInfo.sourceFile.resolveCompletionItem(
this._configOptions, this._importResolver, this._lookUpImport, this._evaluator, this._configOptions,
() => this._buildModuleSymbolsMap(sourceFileInfo), completionItem); this._importResolver,
this._lookUpImport,
this._evaluator,
() => this._buildModuleSymbolsMap(sourceFileInfo),
completionItem
);
} }
performQuickAction(filePath: string, command: string, performQuickAction(filePath: string, command: string, args: any[]): TextEditAction[] | undefined {
args: any[]): TextEditAction[] | undefined {
const sourceFileInfo = this._sourceFileMap.get(filePath); const sourceFileInfo = this._sourceFileMap.get(filePath);
if (!sourceFileInfo) { if (!sourceFileInfo) {
return undefined; return undefined;
@ -826,20 +841,16 @@ export class Program {
this._bindFile(sourceFileInfo); this._bindFile(sourceFileInfo);
return sourceFileInfo.sourceFile.performQuickAction( return sourceFileInfo.sourceFile.performQuickAction(command, args);
command, args);
} }
renameSymbolAtPosition(filePath: string, position: Position, renameSymbolAtPosition(filePath: string, position: Position, newName: string): FileEditAction[] | undefined {
newName: string): FileEditAction[] | undefined {
const sourceFileInfo = this._sourceFileMap.get(filePath); const sourceFileInfo = this._sourceFileMap.get(filePath);
if (!sourceFileInfo) { if (!sourceFileInfo) {
return undefined; return undefined;
} }
const referencesResult = sourceFileInfo.sourceFile.getReferencesForPosition( const referencesResult = sourceFileInfo.sourceFile.getReferencesForPosition(position, true, this._evaluator);
position, true, this._evaluator);
if (!referencesResult) { if (!referencesResult) {
return undefined; return undefined;
@ -851,8 +862,7 @@ export class Program {
if (curSourceFileInfo !== sourceFileInfo) { if (curSourceFileInfo !== sourceFileInfo) {
this._bindFile(curSourceFileInfo); this._bindFile(curSourceFileInfo);
curSourceFileInfo.sourceFile.addReferences(referencesResult, curSourceFileInfo.sourceFile.addReferences(referencesResult, true, this._evaluator);
true, this._evaluator);
} }
} }
} }
@ -975,20 +985,20 @@ export class Program {
return false; return false;
} }
private _isImportAllowed(importer: SourceFileInfo, importResult: ImportResult, private _isImportAllowed(importer: SourceFileInfo, importResult: ImportResult, isImportStubFile: boolean): boolean {
isImportStubFile: boolean): boolean {
let thirdPartyImportAllowed = this._configOptions.useLibraryCodeForTypes; let thirdPartyImportAllowed = this._configOptions.useLibraryCodeForTypes;
if (importResult.importType === ImportType.ThirdParty || if (
(importer.isThirdPartyImport && importResult.importType === ImportType.Local)) { importResult.importType === ImportType.ThirdParty ||
(importer.isThirdPartyImport && importResult.importType === ImportType.Local)
) {
if (this._allowedThirdPartyImports) { if (this._allowedThirdPartyImports) {
if (importResult.isRelative) { if (importResult.isRelative) {
// If it's a relative import, we'll allow it because the // If it's a relative import, we'll allow it because the
// importer was already deemed to be allowed. // importer was already deemed to be allowed.
thirdPartyImportAllowed = true; thirdPartyImportAllowed = true;
} else if (this._allowedThirdPartyImports.some((importName: string) => { } else if (
this._allowedThirdPartyImports.some((importName: string) => {
// If this import name is the one that was explicitly // If this import name is the one that was explicitly
// allowed or is a child of that import name, // allowed or is a child of that import name,
// it's considered allowed. // it's considered allowed.
@ -1001,7 +1011,8 @@ export class Program {
} }
return false; return false;
})) { })
) {
thirdPartyImportAllowed = true; thirdPartyImportAllowed = true;
} }
} }
@ -1018,9 +1029,7 @@ export class Program {
return true; return true;
} }
private _updateSourceFileImports(sourceFileInfo: SourceFileInfo, private _updateSourceFileImports(sourceFileInfo: SourceFileInfo, options: ConfigOptions): SourceFileInfo[] {
options: ConfigOptions): SourceFileInfo[] {
const filesAdded: SourceFileInfo[] = []; const filesAdded: SourceFileInfo[] = [];
// Get the new list of imports and see if it changed from the last // Get the new list of imports and see if it changed from the last
@ -1033,12 +1042,12 @@ export class Program {
if (importResult.isImportFound) { if (importResult.isImportFound) {
if (this._isImportAllowed(sourceFileInfo, importResult, importResult.isStubFile)) { if (this._isImportAllowed(sourceFileInfo, importResult, importResult.isStubFile)) {
if (importResult.resolvedPaths.length > 0) { if (importResult.resolvedPaths.length > 0) {
const filePath = importResult.resolvedPaths[ const filePath = importResult.resolvedPaths[importResult.resolvedPaths.length - 1];
importResult.resolvedPaths.length - 1];
if (filePath) { if (filePath) {
newImportPathMap.set(filePath, { newImportPathMap.set(filePath, {
isTypeshedFile: !!importResult.isTypeshedFile, isTypeshedFile: !!importResult.isTypeshedFile,
isThirdPartyImport: importResult.importType === ImportType.ThirdParty || isThirdPartyImport:
importResult.importType === ImportType.ThirdParty ||
(sourceFileInfo.isThirdPartyImport && importResult.importType === ImportType.Local) (sourceFileInfo.isThirdPartyImport && importResult.importType === ImportType.Local)
}); });
} }
@ -1049,15 +1058,18 @@ export class Program {
if (this._isImportAllowed(sourceFileInfo, importResult, implicitImport.isStubFile)) { if (this._isImportAllowed(sourceFileInfo, importResult, implicitImport.isStubFile)) {
newImportPathMap.set(implicitImport.path, { newImportPathMap.set(implicitImport.path, {
isTypeshedFile: !!importResult.isTypeshedFile, isTypeshedFile: !!importResult.isTypeshedFile,
isThirdPartyImport: importResult.importType === ImportType.ThirdParty || isThirdPartyImport:
importResult.importType === ImportType.ThirdParty ||
(sourceFileInfo.isThirdPartyImport && importResult.importType === ImportType.Local) (sourceFileInfo.isThirdPartyImport && importResult.importType === ImportType.Local)
}); });
} }
}); });
} else if (options.verboseOutput) { } else if (options.verboseOutput) {
if (!sourceFileInfo.isTypeshedFile || options.diagnosticSettings.reportTypeshedErrors) { if (!sourceFileInfo.isTypeshedFile || options.diagnosticSettings.reportTypeshedErrors) {
this._console.log(`Could not import '${ importResult.importName }' ` + this._console.log(
`in file '${ sourceFileInfo.sourceFile.getFilePath() }'`); `Could not import '${importResult.importName}' ` +
`in file '${sourceFileInfo.sourceFile.getFilePath()}'`
);
if (importResult.importFailureInfo) { if (importResult.importFailureInfo) {
importResult.importFailureInfo.forEach(diag => { importResult.importFailureInfo.forEach(diag => {
this._console.log(` ${diag}`); this._console.log(` ${diag}`);
@ -1074,7 +1086,8 @@ export class Program {
// A previous import was removed. // A previous import was removed.
if (!newImportPathMap.has(oldFilePath)) { if (!newImportPathMap.has(oldFilePath)) {
importInfo.importedBy = importInfo.importedBy.filter( importInfo.importedBy = importInfo.importedBy.filter(
fi => fi.sourceFile.getFilePath() !== sourceFileInfo.sourceFile.getFilePath()); fi => fi.sourceFile.getFilePath() !== sourceFileInfo.sourceFile.getFilePath()
);
} else { } else {
updatedImportMap.set(oldFilePath, importInfo); updatedImportMap.set(oldFilePath, importInfo);
} }
@ -1091,8 +1104,11 @@ export class Program {
} else { } else {
const sourceFile = new SourceFile( const sourceFile = new SourceFile(
this._fs, this._fs,
importPath, importInfo.isTypeshedFile, importPath,
importInfo.isThirdPartyImport, this._console); importInfo.isTypeshedFile,
importInfo.isThirdPartyImport,
this._console
);
importedFileInfo = { importedFileInfo = {
sourceFile, sourceFile,
isTracked: false, isTracked: false,
@ -1127,8 +1143,7 @@ export class Program {
sourceFileInfo.builtinsImport = undefined; sourceFileInfo.builtinsImport = undefined;
const builtinsImport = sourceFileInfo.sourceFile.getBuiltinsImport(); const builtinsImport = sourceFileInfo.sourceFile.getBuiltinsImport();
if (builtinsImport) { if (builtinsImport) {
const resolvedBuiltinsPath = builtinsImport.resolvedPaths[ const resolvedBuiltinsPath = builtinsImport.resolvedPaths[builtinsImport.resolvedPaths.length - 1];
builtinsImport.resolvedPaths.length - 1];
sourceFileInfo.builtinsImport = this._sourceFileMap.get(resolvedBuiltinsPath); sourceFileInfo.builtinsImport = this._sourceFileMap.get(resolvedBuiltinsPath);
} }

35
server/src/analyzer/pythonPathUtils.ts
View File

@ -11,8 +11,14 @@ import * as child_process from 'child_process';
import { ConfigOptions } from '../common/configOptions'; import { ConfigOptions } from '../common/configOptions';
import * as pathConsts from '../common/pathConsts'; import * as pathConsts from '../common/pathConsts';
import { combinePaths, ensureTrailingDirectorySeparator, getDirectoryPath, import {
getFileSystemEntries, isDirectory, normalizePath } from '../common/pathUtils'; combinePaths,
ensureTrailingDirectorySeparator,
getDirectoryPath,
getFileSystemEntries,
isDirectory,
normalizePath
} from '../common/pathUtils';
import { VirtualFileSystem } from '../common/vfs'; import { VirtualFileSystem } from '../common/vfs';
const cachedSearchPaths = new Map<string, string[]>(); const cachedSearchPaths = new Map<string, string[]>();
@ -23,8 +29,7 @@ export function getTypeShedFallbackPath(fs: VirtualFileSystem) {
return undefined; return undefined;
} }
moduleDirectory = getDirectoryPath(ensureTrailingDirectorySeparator( moduleDirectory = getDirectoryPath(ensureTrailingDirectorySeparator(normalizePath(moduleDirectory)));
normalizePath(moduleDirectory)));
const typeshedPath = combinePaths(moduleDirectory, pathConsts.typeshedFallback); const typeshedPath = combinePaths(moduleDirectory, pathConsts.typeshedFallback);
if (fs.existsSync(typeshedPath)) { if (fs.existsSync(typeshedPath)) {
@ -45,9 +50,12 @@ export function getTypeshedSubdirectory(typeshedPath: string, isStdLib: boolean)
return combinePaths(typeshedPath, isStdLib ? 'stdlib' : 'third_party'); return combinePaths(typeshedPath, isStdLib ? 'stdlib' : 'third_party');
} }
export function findPythonSearchPaths(fs: VirtualFileSystem, configOptions: ConfigOptions, export function findPythonSearchPaths(
venv: string | undefined, importFailureInfo: string[]): string[] | undefined { fs: VirtualFileSystem,
configOptions: ConfigOptions,
venv: string | undefined,
importFailureInfo: string[]
): string[] | undefined {
importFailureInfo.push('Finding python search paths'); importFailureInfo.push('Finding python search paths');
let venvPath: string | undefined; let venvPath: string | undefined;
@ -109,10 +117,11 @@ export function findPythonSearchPaths(fs: VirtualFileSystem, configOptions: Conf
return getPythonPathFromPythonInterpreter(fs, configOptions.pythonPath, importFailureInfo); return getPythonPathFromPythonInterpreter(fs, configOptions.pythonPath, importFailureInfo);
} }
export function getPythonPathFromPythonInterpreter(fs: VirtualFileSystem, export function getPythonPathFromPythonInterpreter(
fs: VirtualFileSystem,
interpreterPath: string | undefined, interpreterPath: string | undefined,
importFailureInfo: string[]): string[] { importFailureInfo: string[]
): string[] {
const searchKey = interpreterPath || ''; const searchKey = interpreterPath || '';
// If we've seen this request before, return the cached results. // If we've seen this request before, return the cached results.
@ -137,12 +146,10 @@ export function getPythonPathFromPythonInterpreter(fs: VirtualFileSystem,
if (interpreterPath) { if (interpreterPath) {
importFailureInfo.push(`Executing interpreter at '${interpreterPath}'`); importFailureInfo.push(`Executing interpreter at '${interpreterPath}'`);
execOutput = child_process.execFileSync( execOutput = child_process.execFileSync(interpreterPath, commandLineArgs, { encoding: 'utf8' });
interpreterPath, commandLineArgs, { encoding: 'utf8' });
} else { } else {
importFailureInfo.push(`Executing python interpreter`); importFailureInfo.push(`Executing python interpreter`);
execOutput = child_process.execFileSync( execOutput = child_process.execFileSync('python', commandLineArgs, { encoding: 'utf8' });
'python', commandLineArgs, { encoding: 'utf8' });
} }
// Parse the execOutput. It should be a JSON-encoded array of paths. // Parse the execOutput. It should be a JSON-encoded array of paths.

14
server/src/analyzer/scope.ts
View File

@ -103,9 +103,11 @@ export class Scope {
return symbol; return symbol;
} }
private _lookUpSymbolRecursiveInternal(name: string, isOutsideCallerModule: boolean, private _lookUpSymbolRecursiveInternal(
isBeyondExecutionScope: boolean): SymbolWithScope | undefined { name: string,
isOutsideCallerModule: boolean,
isBeyondExecutionScope: boolean
): SymbolWithScope | undefined {
const symbol = this.symbolTable.get(name); const symbol = this.symbolTable.get(name);
if (symbol) { if (symbol) {
@ -127,9 +129,11 @@ export class Scope {
// If our recursion is about to take us outside the scope of the current // If our recursion is about to take us outside the scope of the current
// module (i.e. into a built-in scope), indicate as such with the second // module (i.e. into a built-in scope), indicate as such with the second
// parameter. // parameter.
return this.parent._lookUpSymbolRecursiveInternal(name, return this.parent._lookUpSymbolRecursiveInternal(
name,
isOutsideCallerModule || this.type === ScopeType.Module, isOutsideCallerModule || this.type === ScopeType.Module,
isBeyondExecutionScope || this.isIndependentlyExecutable()); isBeyondExecutionScope || this.isIndependentlyExecutable()
);
} }
return undefined; return undefined;

151
server/src/analyzer/service.ts
View File

@ -18,9 +18,16 @@ import { Diagnostic } from '../common/diagnostic';
import { FileDiagnostics } from '../common/diagnosticSink'; import { FileDiagnostics } from '../common/diagnosticSink';
import { FileEditAction, TextEditAction } from '../common/editAction'; import { FileEditAction, TextEditAction } from '../common/editAction';
import { import {
combinePaths, FileSpec, forEachAncestorDirectory, getDirectoryPath, combinePaths,
getFileName, getFileSpec, getFileSystemEntries, isDirectory, FileSpec,
normalizePath, stripFileExtension forEachAncestorDirectory,
getDirectoryPath,
getFileName,
getFileSpec,
getFileSystemEntries,
isDirectory,
normalizePath,
stripFileExtension
} from '../common/pathUtils'; } from '../common/pathUtils';
import { DocumentRange, Position, Range } from '../common/textRange'; import { DocumentRange, Position, Range } from '../common/textRange';
import { Duration, timingStats } from '../common/timing'; import { Duration, timingStats } from '../common/timing';
@ -70,9 +77,13 @@ export class AnalyzerService {
private _requireTrackedFileUpdate = true; private _requireTrackedFileUpdate = true;
private _lastUserInteractionTime = Date.now(); private _lastUserInteractionTime = Date.now();
constructor(instanceName: string, fs: VirtualFileSystem, console?: ConsoleInterface, constructor(
importResolverFactory?: ImportResolverFactory, configOptions?: ConfigOptions) { instanceName: string,
fs: VirtualFileSystem,
console?: ConsoleInterface,
importResolverFactory?: ImportResolverFactory,
configOptions?: ConfigOptions
) {
this._instanceName = instanceName; this._instanceName = instanceName;
this._console = console || new StandardConsole(); this._console = console || new StandardConsole();
this._configOptions = configOptions ?? new ConfigOptions(process.cwd()); this._configOptions = configOptions ?? new ConfigOptions(process.cwd());
@ -84,7 +95,13 @@ export class AnalyzerService {
} }
clone(instanceName: string): AnalyzerService { clone(instanceName: string): AnalyzerService {
return new AnalyzerService(instanceName, this._fs, this._console, this._importResolverFactory, this._configOptions); return new AnalyzerService(
instanceName,
this._fs,
this._console,
this._importResolverFactory,
this._configOptions
);
} }
dispose() { dispose() {
@ -113,8 +130,9 @@ export class AnalyzerService {
this._program.setConfigOptions(this._configOptions); this._program.setConfigOptions(this._configOptions);
this._typeStubTargetImportName = commandLineOptions.typeStubTargetImportName; this._typeStubTargetImportName = commandLineOptions.typeStubTargetImportName;
this._executionRootPath = normalizePath(combinePaths( this._executionRootPath = normalizePath(
commandLineOptions.executionRoot, this._configOptions.projectRoot)); combinePaths(commandLineOptions.executionRoot, this._configOptions.projectRoot)
);
this._applyConfigOptions(); this._applyConfigOptions();
} }
@ -135,15 +153,15 @@ export class AnalyzerService {
this._scheduleReanalysis(false); this._scheduleReanalysis(false);
} }
getDefinitionForPosition(filePath: string, position: Position): getDefinitionForPosition(filePath: string, position: Position): DocumentRange[] | undefined {
DocumentRange[] | undefined {
return this._program.getDefinitionsForPosition(filePath, position); return this._program.getDefinitionsForPosition(filePath, position);
} }
getReferencesForPosition(filePath: string, position: Position, getReferencesForPosition(
includeDeclaration: boolean): DocumentRange[] | undefined { filePath: string,
position: Position,
includeDeclaration: boolean
): DocumentRange[] | undefined {
return this._program.getReferencesForPosition(filePath, position, includeDeclaration); return this._program.getReferencesForPosition(filePath, position, includeDeclaration);
} }
@ -155,21 +173,15 @@ export class AnalyzerService {
this._program.addSymbolsForWorkspace(symbolList, query); this._program.addSymbolsForWorkspace(symbolList, query);
} }
getHoverForPosition(filePath: string, position: Position): getHoverForPosition(filePath: string, position: Position): HoverResults | undefined {
HoverResults | undefined {
return this._program.getHoverForPosition(filePath, position); return this._program.getHoverForPosition(filePath, position);
} }
getSignatureHelpForPosition(filePath: string, position: Position): getSignatureHelpForPosition(filePath: string, position: Position): SignatureHelpResults | undefined {
SignatureHelpResults | undefined {
return this._program.getSignatureHelpForPosition(filePath, position); return this._program.getSignatureHelpForPosition(filePath, position);
} }
getCompletionsForPosition(filePath: string, position: Position, getCompletionsForPosition(filePath: string, position: Position, workspacePath: string): CompletionList | undefined {
workspacePath: string): CompletionList | undefined {
return this._program.getCompletionsForPosition(filePath, position, workspacePath); return this._program.getCompletionsForPosition(filePath, position, workspacePath);
} }
@ -181,9 +193,7 @@ export class AnalyzerService {
return this._program.performQuickAction(filePath, command, args); return this._program.performQuickAction(filePath, command, args);
} }
renameSymbolAtPosition(filePath: string, position: Position, renameSymbolAtPosition(filePath: string, position: Position, newName: string): FileEditAction[] | undefined {
newName: string): FileEditAction[] | undefined {
return this._program.renameSymbolAtPosition(filePath, position, newName); return this._program.renameSymbolAtPosition(filePath, position, newName);
} }
@ -251,8 +261,10 @@ export class AnalyzerService {
// If the config file path was specified, determine whether it's // If the config file path was specified, determine whether it's
// a directory (in which case the default config file name is assumed) // a directory (in which case the default config file name is assumed)
// or a file. // or a file.
configFilePath = combinePaths(commandLineOptions.executionRoot, configFilePath = combinePaths(
normalizePath(commandLineOptions.configFilePath)); commandLineOptions.executionRoot,
normalizePath(commandLineOptions.configFilePath)
);
if (!this._fs.existsSync(configFilePath)) { if (!this._fs.existsSync(configFilePath)) {
this._console.log(`Configuration file not found at ${configFilePath}.`); this._console.log(`Configuration file not found at ${configFilePath}.`);
configFilePath = commandLineOptions.executionRoot; configFilePath = commandLineOptions.executionRoot;
@ -328,12 +340,14 @@ export class AnalyzerService {
} }
const reportDuplicateSetting = (settingName: string) => { const reportDuplicateSetting = (settingName: string) => {
const settingSource = commandLineOptions.fromVsCodeExtension ? const settingSource = commandLineOptions.fromVsCodeExtension
'the VS Code settings' : 'a command-line option'; ? 'the VS Code settings'
: 'a command-line option';
this._console.log( this._console.log(
`The ${settingName} has been specified in both the config file and ` + `The ${settingName} has been specified in both the config file and ` +
`${settingSource}. The value in the config file (${configOptions.venvPath}) ` + `${settingSource}. The value in the config file (${configOptions.venvPath}) ` +
`will take precedence`); `will take precedence`
);
}; };
// Apply the command-line options if the corresponding // Apply the command-line options if the corresponding
@ -348,8 +362,9 @@ export class AnalyzerService {
} }
if (commandLineOptions.pythonPath) { if (commandLineOptions.pythonPath) {
this._console.log(`Setting pythonPath for service "${ this._instanceName }": ` + this._console.log(
`"${ commandLineOptions.pythonPath }"`); `Setting pythonPath for service "${this._instanceName}": ` + `"${commandLineOptions.pythonPath}"`
);
configOptions.pythonPath = commandLineOptions.pythonPath; configOptions.pythonPath = commandLineOptions.pythonPath;
} }
@ -374,8 +389,7 @@ export class AnalyzerService {
// or inconsistent information. // or inconsistent information.
if (configOptions.venvPath) { if (configOptions.venvPath) {
if (!this._fs.existsSync(configOptions.venvPath) || !isDirectory(this._fs, configOptions.venvPath)) { if (!this._fs.existsSync(configOptions.venvPath) || !isDirectory(this._fs, configOptions.venvPath)) {
this._console.log( this._console.log(`venvPath ${configOptions.venvPath} is not a valid directory.`);
`venvPath ${ configOptions.venvPath } is not a valid directory.`);
} }
// venvPath without defaultVenv means it won't do anything while resolveImport. // venvPath without defaultVenv means it won't do anything while resolveImport.
@ -388,13 +402,15 @@ export class AnalyzerService {
if (!this._fs.existsSync(fullVenvPath) || !isDirectory(this._fs, fullVenvPath)) { if (!this._fs.existsSync(fullVenvPath) || !isDirectory(this._fs, fullVenvPath)) {
this._console.log( this._console.log(
`venv ${configOptions.defaultVenv} subdirectory not found ` + `venv ${configOptions.defaultVenv} subdirectory not found ` +
`in venv path ${ configOptions.venvPath }.`); `in venv path ${configOptions.venvPath}.`
);
} else { } else {
const importFailureInfo: string[] = []; const importFailureInfo: string[] = [];
if (findPythonSearchPaths(this._fs, configOptions, undefined, importFailureInfo) === undefined) { if (findPythonSearchPaths(this._fs, configOptions, undefined, importFailureInfo) === undefined) {
this._console.log( this._console.log(
`site-packages directory cannot be located for venvPath ` + `site-packages directory cannot be located for venvPath ` +
`${ configOptions.venvPath } and venv ${ configOptions.defaultVenv }.`); `${configOptions.venvPath} and venv ${configOptions.defaultVenv}.`
);
if (configOptions.verboseOutput) { if (configOptions.verboseOutput) {
importFailureInfo.forEach(diag => { importFailureInfo.forEach(diag => {
@ -406,16 +422,18 @@ export class AnalyzerService {
} }
} else { } else {
const importFailureInfo: string[] = []; const importFailureInfo: string[] = [];
const pythonPaths = getPythonPathFromPythonInterpreter(this._fs, configOptions.pythonPath, importFailureInfo); const pythonPaths = getPythonPathFromPythonInterpreter(
this._fs,
configOptions.pythonPath,
importFailureInfo
);
if (pythonPaths.length === 0) { if (pythonPaths.length === 0) {
if (configOptions.verboseOutput) { if (configOptions.verboseOutput) {
this._console.log( this._console.log(`No search paths found for configured python interpreter.`);
`No search paths found for configured python interpreter.`);
} }
} else { } else {
if (configOptions.verboseOutput) { if (configOptions.verboseOutput) {
this._console.log( this._console.log(`Search paths found for configured python interpreter:`);
`Search paths found for configured python interpreter:`);
pythonPaths.forEach(path => { pythonPaths.forEach(path => {
this._console.log(` ${path}`); this._console.log(` ${path}`);
}); });
@ -424,8 +442,7 @@ export class AnalyzerService {
if (configOptions.verboseOutput) { if (configOptions.verboseOutput) {
if (importFailureInfo.length > 0) { if (importFailureInfo.length > 0) {
this._console.log( this._console.log(`When attempting to get search paths from python interpreter:`);
`When attempting to get search paths from python interpreter:`);
importFailureInfo.forEach(diag => { importFailureInfo.forEach(diag => {
this._console.log(` ${diag}`); this._console.log(` ${diag}`);
}); });
@ -436,22 +453,22 @@ export class AnalyzerService {
// Is there a reference to a venv? If so, there needs to be a valid venvPath. // Is there a reference to a venv? If so, there needs to be a valid venvPath.
if (configOptions.defaultVenv || configOptions.executionEnvironments.find(e => !!e.venv)) { if (configOptions.defaultVenv || configOptions.executionEnvironments.find(e => !!e.venv)) {
if (!configOptions.venvPath) { if (!configOptions.venvPath) {
this._console.log( this._console.log(`venvPath not specified, so venv settings will be ignored.`);
`venvPath not specified, so venv settings will be ignored.`);
} }
} }
if (configOptions.typeshedPath) { if (configOptions.typeshedPath) {
if (!this._fs.existsSync(configOptions.typeshedPath) || !isDirectory(this._fs, configOptions.typeshedPath)) { if (
this._console.log( !this._fs.existsSync(configOptions.typeshedPath) ||
`typeshedPath ${ configOptions.typeshedPath } is not a valid directory.`); !isDirectory(this._fs, configOptions.typeshedPath)
) {
this._console.log(`typeshedPath ${configOptions.typeshedPath} is not a valid directory.`);
} }
} }
if (configOptions.typingsPath) { if (configOptions.typingsPath) {
if (!this._fs.existsSync(configOptions.typingsPath) || !isDirectory(this._fs, configOptions.typingsPath)) { if (!this._fs.existsSync(configOptions.typingsPath) || !isDirectory(this._fs, configOptions.typingsPath)) {
this._console.log( this._console.log(`typingsPath ${configOptions.typingsPath} is not a valid directory.`);
`typingsPath ${ configOptions.typingsPath } is not a valid directory.`);
} }
} }
@ -461,8 +478,7 @@ export class AnalyzerService {
writeTypeStub() { writeTypeStub() {
const typingsPath = this._configOptions.typingsPath; const typingsPath = this._configOptions.typingsPath;
if (!this._typeStubTargetPath || !this._typeStubTargetImportName) { if (!this._typeStubTargetPath || !this._typeStubTargetImportName) {
const errMsg = `Import '${ this._typeStubTargetImportName }'` + const errMsg = `Import '${this._typeStubTargetImportName}'` + ` could not be resolved`;
` could not be resolved`;
this._console.error(errMsg); this._console.error(errMsg);
throw new Error(errMsg); throw new Error(errMsg);
} }
@ -479,8 +495,7 @@ export class AnalyzerService {
if (typeStubInputTargetParts[0].length === 0) { if (typeStubInputTargetParts[0].length === 0) {
// We should never get here because the import resolution // We should never get here because the import resolution
// would have failed. // would have failed.
const errMsg = `Import '${ this._typeStubTargetImportName }'` + const errMsg = `Import '${this._typeStubTargetImportName}'` + ` could not be resolved`;
` could not be resolved`;
this._console.error(errMsg); this._console.error(errMsg);
throw new Error(errMsg); throw new Error(errMsg);
} }
@ -587,8 +602,7 @@ export class AnalyzerService {
const fileMap = new Map<string, string>(); const fileMap = new Map<string, string>();
timingStats.findFilesTime.timeOperation(() => { timingStats.findFilesTime.timeOperation(() => {
const matchedFiles = this._matchFiles(this._configOptions.include, const matchedFiles = this._matchFiles(this._configOptions.include, this._configOptions.exclude);
this._configOptions.exclude);
for (const file of matchedFiles) { for (const file of matchedFiles) {
fileMap.set(file, file); fileMap.set(file, file);
@ -614,16 +628,14 @@ export class AnalyzerService {
importedSymbols: [] importedSymbols: []
}; };
const importResult = this._importResolver.resolveImport( const importResult = this._importResolver.resolveImport('', execEnv, moduleDescriptor);
'', execEnv, moduleDescriptor);
if (importResult.isImportFound) { if (importResult.isImportFound) {
const filesToImport: string[] = []; const filesToImport: string[] = [];
// Namespace packages resolve to a directory name, so // Namespace packages resolve to a directory name, so
// don't include those. // don't include those.
const resolvedPath = importResult.resolvedPaths[ const resolvedPath = importResult.resolvedPaths[importResult.resolvedPaths.length - 1];
importResult.resolvedPaths.length - 1];
// Get the directory that contains the root package. // Get the directory that contains the root package.
let targetPath = getDirectoryPath(resolvedPath); let targetPath = getDirectoryPath(resolvedPath);
@ -647,7 +659,8 @@ export class AnalyzerService {
this._typeStubTargetIsSingleFile = false; this._typeStubTargetIsSingleFile = false;
} else { } else {
filesToImport.push(resolvedPath); filesToImport.push(resolvedPath);
this._typeStubTargetIsSingleFile = importResult.resolvedPaths.length === 1 && this._typeStubTargetIsSingleFile =
importResult.resolvedPaths.length === 1 &&
stripFileExtension(getFileName(importResult.resolvedPaths[0])) !== '__init__'; stripFileExtension(getFileName(importResult.resolvedPaths[0])) !== '__init__';
} }
@ -673,8 +686,7 @@ export class AnalyzerService {
if (fileList.length === 0) { if (fileList.length === 0) {
this._console.log(`No source files found.`); this._console.log(`No source files found.`);
} else { } else {
this._console.log(`Found ${ fileList.length } ` + this._console.log(`Found ${fileList.length} ` + `source ${fileList.length === 1 ? 'file' : 'files'}`);
`source ${ fileList.length === 1 ? 'file' : 'files' }`);
} }
} }
@ -798,8 +810,7 @@ export class AnalyzerService {
this._removeConfigFileWatcher(); this._removeConfigFileWatcher();
if (this._configFilePath) { if (this._configFilePath) {
this._configFileWatcher = this._fs.createFileSystemWatcher([this._configFilePath], this._configFileWatcher = this._fs.createFileSystemWatcher([this._configFilePath], 'all', event => {
'all', event => {
if (this._verboseOutput) { if (this._verboseOutput) {
this._console.log(`Received fs event '${event}' for config file`); this._console.log(`Received fs event '${event}' for config file`);
} }
@ -881,7 +892,8 @@ export class AnalyzerService {
const timeUntilNextAnalysisInMs = Math.max( const timeUntilNextAnalysisInMs = Math.max(
minBackoffTimeInMs - timeSinceLastUserInteractionInMs, minBackoffTimeInMs - timeSinceLastUserInteractionInMs,
minTimeBetweenAnalysisPassesInMs); minTimeBetweenAnalysisPassesInMs
);
// Schedule a new timer. // Schedule a new timer.
this._analyzeTimer = setTimeout(() => { this._analyzeTimer = setTimeout(() => {
@ -943,7 +955,8 @@ export class AnalyzerService {
} }
} }
} catch (e) { } catch (e) {
const message: string = (e.stack ? e.stack.toString() : undefined) || const message: string =
(e.stack ? e.stack.toString() : undefined) ||
(typeof e.message === 'string' ? e.message : undefined) || (typeof e.message === 'string' ? e.message : undefined) ||
JSON.stringify(e); JSON.stringify(e);
this._console.log('Error performing analysis: ' + message); this._console.log('Error performing analysis: ' + message);

293
server/src/analyzer/sourceFile.ts
View File

@ -9,10 +9,7 @@
import { CompletionItem, CompletionList, DocumentSymbol, SymbolInformation } from 'vscode-languageserver'; import { CompletionItem, CompletionList, DocumentSymbol, SymbolInformation } from 'vscode-languageserver';
import { import { ConfigOptions, ExecutionEnvironment, getDefaultDiagnosticSettings } from '../common/configOptions';
ConfigOptions, ExecutionEnvironment,
getDefaultDiagnosticSettings
} from '../common/configOptions';
import { ConsoleInterface, StandardConsole } from '../common/console'; import { ConsoleInterface, StandardConsole } from '../common/console';
import { assert } from '../common/debug'; import { assert } from '../common/debug';
import { Diagnostic, DiagnosticCategory } from '../common/diagnostic'; import { Diagnostic, DiagnosticCategory } from '../common/diagnostic';
@ -142,9 +139,13 @@ export class SourceFile {
readonly fileSystem: VirtualFileSystem; readonly fileSystem: VirtualFileSystem;
constructor(fs: VirtualFileSystem, filePath: string, isTypeshedStubFile: boolean, constructor(
isThirdPartyImport: boolean, console?: ConsoleInterface) { fs: VirtualFileSystem,
filePath: string,
isTypeshedStubFile: boolean,
isThirdPartyImport: boolean,
console?: ConsoleInterface
) {
this.fileSystem = fs; this.fileSystem = fs;
this._console = console || new StandardConsole(); this._console = console || new StandardConsole();
this._filePath = filePath; this._filePath = filePath;
@ -152,18 +153,19 @@ export class SourceFile {
this._isTypeshedStubFile = isTypeshedStubFile; this._isTypeshedStubFile = isTypeshedStubFile;
this._isThirdPartyImport = isThirdPartyImport; this._isThirdPartyImport = isThirdPartyImport;
const fileName = getFileName(filePath); const fileName = getFileName(filePath);
this._isTypingStubFile = this._isStubFile && ( this._isTypingStubFile =
fileName === 'typing.pyi' || fileName === 'typing_extensions.pyi'); this._isStubFile && (fileName === 'typing.pyi' || fileName === 'typing_extensions.pyi');
this._isBuiltInStubFile = false; this._isBuiltInStubFile = false;
if (this._isStubFile) { if (this._isStubFile) {
if (this._filePath.endsWith(normalizeSlashes('/collections/__init__.pyi')) || if (
this._filePath.endsWith(normalizeSlashes('/collections/__init__.pyi')) ||
fileName === 'builtins.pyi' || fileName === 'builtins.pyi' ||
fileName === '_importlib_modulespec.pyi' || fileName === '_importlib_modulespec.pyi' ||
fileName === 'dataclasses.pyi' || fileName === 'dataclasses.pyi' ||
fileName === 'abc.pyi' || fileName === 'abc.pyi' ||
fileName === 'enum.pyi') { fileName === 'enum.pyi'
) {
this._isBuiltInStubFile = true; this._isBuiltInStubFile = true;
} }
} }
@ -184,9 +186,7 @@ export class SourceFile {
// Returns a list of cached diagnostics from the latest analysis job. // Returns a list of cached diagnostics from the latest analysis job.
// If the prevVersion is specified, the method returns undefined if // If the prevVersion is specified, the method returns undefined if
// the diagnostics haven't changed. // the diagnostics haven't changed.
getDiagnostics(options: ConfigOptions, prevDiagnosticVersion?: number): getDiagnostics(options: ConfigOptions, prevDiagnosticVersion?: number): Diagnostic[] | undefined {
Diagnostic[] | undefined {
if (this._diagnosticVersion === prevDiagnosticVersion) { if (this._diagnosticVersion === prevDiagnosticVersion) {
return undefined; return undefined;
} }
@ -200,15 +200,11 @@ export class SourceFile {
} }
let diagList: Diagnostic[] = []; let diagList: Diagnostic[] = [];
diagList = diagList.concat( diagList = diagList.concat(this._parseDiagnostics, this._bindDiagnostics, this._checkerDiagnostics);
this._parseDiagnostics,
this._bindDiagnostics,
this._checkerDiagnostics);
// Filter the diagnostics based on "type: ignore" lines. // Filter the diagnostics based on "type: ignore" lines.
if (options.diagnosticSettings.enableTypeIgnoreComments) { if (options.diagnosticSettings.enableTypeIgnoreComments) {
const typeIgnoreLines = this._parseResults ? const typeIgnoreLines = this._parseResults ? this._parseResults.tokenizerOutput.typeIgnoreLines : {};
this._parseResults.tokenizerOutput.typeIgnoreLines : {};
if (Object.keys(typeIgnoreLines).length > 0) { if (Object.keys(typeIgnoreLines).length > 0) {
diagList = diagList.filter(d => { diagList = diagList.filter(d => {
for (let line = d.range.start.line; line <= d.range.end.line; line++) { for (let line = d.range.start.line; line <= d.range.end.line; line++) {
@ -223,19 +219,34 @@ export class SourceFile {
} }
if (options.diagnosticSettings.reportImportCycles !== 'none' && this._circularDependencies.length > 0) { if (options.diagnosticSettings.reportImportCycles !== 'none' && this._circularDependencies.length > 0) {
const category = options.diagnosticSettings.reportImportCycles === 'warning' ? const category =
DiagnosticCategory.Warning : DiagnosticCategory.Error; options.diagnosticSettings.reportImportCycles === 'warning'
? DiagnosticCategory.Warning
: DiagnosticCategory.Error;
this._circularDependencies.forEach(cirDep => { this._circularDependencies.forEach(cirDep => {
diagList.push(new Diagnostic(category, 'Cycle detected in import chain\n' + diagList.push(
cirDep.getPaths().map(path => ' ' + path).join('\n'), getEmptyRange())); new Diagnostic(
category,
'Cycle detected in import chain\n' +
cirDep
.getPaths()
.map(path => ' ' + path)
.join('\n'),
getEmptyRange()
)
);
}); });
} }
if (this._hitMaxImportDepth !== undefined) { if (this._hitMaxImportDepth !== undefined) {
diagList.push(new Diagnostic(DiagnosticCategory.Error, diagList.push(
new Diagnostic(
DiagnosticCategory.Error,
`Import chain depth exceeded ${this._hitMaxImportDepth}`, `Import chain depth exceeded ${this._hitMaxImportDepth}`,
getEmptyRange())); getEmptyRange()
)
);
} }
if (this._isTypeshedStubFile) { if (this._isTypeshedStubFile) {
@ -245,8 +256,7 @@ export class SourceFile {
// Convert all the errors to warnings. // Convert all the errors to warnings.
diagList = diagList.map(diag => { diagList = diagList.map(diag => {
if (diag.category === DiagnosticCategory.Error) { if (diag.category === DiagnosticCategory.Error) {
return new Diagnostic(DiagnosticCategory.Warning, return new Diagnostic(DiagnosticCategory.Warning, diag.message, diag.range);
diag.message, diag.range);
} }
return diag; return diag;
}); });
@ -471,24 +481,30 @@ export class SourceFile {
// Resolve imports. // Resolve imports.
timingStats.resolveImportsTime.timeOperation(() => { timingStats.resolveImportsTime.timeOperation(() => {
[this._imports, this._builtinsImport, this._typingModulePath, this._collectionsModulePath] = [
this._resolveImports(importResolver, parseResults.importedModules, execEnvironment); this._imports,
this._builtinsImport,
this._typingModulePath,
this._collectionsModulePath
] = this._resolveImports(importResolver, parseResults.importedModules, execEnvironment);
this._parseDiagnostics = diagSink.fetchAndClear(); this._parseDiagnostics = diagSink.fetchAndClear();
}); });
// Is this file in a "strict" path? // Is this file in a "strict" path?
const useStrict = configOptions.strict.find( const useStrict =
strictFileSpec => strictFileSpec.regExp.test(this._filePath)) !== undefined; configOptions.strict.find(strictFileSpec => strictFileSpec.regExp.test(this._filePath)) !== undefined;
this._diagnosticSettings = CommentUtils.getFileLevelDirectives( this._diagnosticSettings = CommentUtils.getFileLevelDirectives(
this._parseResults.tokenizerOutput.tokens, configOptions.diagnosticSettings, this._parseResults.tokenizerOutput.tokens,
useStrict); configOptions.diagnosticSettings,
useStrict
);
} catch (e) { } catch (e) {
const message: string = (e.stack ? e.stack.toString() : undefined) || const message: string =
(e.stack ? e.stack.toString() : undefined) ||
(typeof e.message === 'string' ? e.message : undefined) || (typeof e.message === 'string' ? e.message : undefined) ||
JSON.stringify(e); JSON.stringify(e);
this._console.log( this._console.log(`An internal error occurred while parsing ${this.getFilePath()}: ` + message);
`An internal error occurred while parsing ${ this.getFilePath() }: ` + message);
// Create dummy parse results. // Create dummy parse results.
this._parseResults = { this._parseResults = {
@ -503,7 +519,7 @@ export class SourceFile {
typeIgnoreLines: {}, typeIgnoreLines: {},
predominantEndOfLineSequence: '\n', predominantEndOfLineSequence: '\n',
predominantTabSequence: ' ', predominantTabSequence: ' ',
predominantSingleQuoteCharacter: '\'' predominantSingleQuoteCharacter: "'"
}, },
containsWildcardImport: false containsWildcardImport: false
}; };
@ -525,41 +541,47 @@ export class SourceFile {
return true; return true;
} }
getDefinitionsForPosition(position: Position, getDefinitionsForPosition(position: Position, evaluator: TypeEvaluator): DocumentRange[] | undefined {
evaluator: TypeEvaluator): DocumentRange[] | undefined {
// If we have no completed analysis job, there's nothing to do. // If we have no completed analysis job, there's nothing to do.
if (!this._parseResults) { if (!this._parseResults) {
return undefined; return undefined;
} }
return DefinitionProvider.getDefinitionsForPosition( return DefinitionProvider.getDefinitionsForPosition(this._parseResults, position, evaluator);
this._parseResults, position, evaluator);
} }
getReferencesForPosition(position: Position, includeDeclaration: boolean, getReferencesForPosition(
evaluator: TypeEvaluator): ReferencesResult | undefined { position: Position,
includeDeclaration: boolean,
evaluator: TypeEvaluator
): ReferencesResult | undefined {
// If we have no completed analysis job, there's nothing to do. // If we have no completed analysis job, there's nothing to do.
if (!this._parseResults) { if (!this._parseResults) {
return undefined; return undefined;
} }
return ReferencesProvider.getReferencesForPosition(this._parseResults, return ReferencesProvider.getReferencesForPosition(
this._filePath, position, includeDeclaration, evaluator); this._parseResults,
this._filePath,
position,
includeDeclaration,
evaluator
);
} }
addReferences(referencesResult: ReferencesResult, includeDeclaration: boolean, addReferences(referencesResult: ReferencesResult, includeDeclaration: boolean, evaluator: TypeEvaluator): void {
evaluator: TypeEvaluator): void {
// If we have no completed analysis job, there's nothing to do. // If we have no completed analysis job, there's nothing to do.
if (!this._parseResults) { if (!this._parseResults) {
return; return;
} }
ReferencesProvider.addReferences( ReferencesProvider.addReferences(
this._parseResults, this._filePath, referencesResult, includeDeclaration, this._parseResults,
evaluator); this._filePath,
referencesResult,
includeDeclaration,
evaluator
);
} }
addHierarchicalSymbolsForDocument(symbolList: DocumentSymbol[], evaluator: TypeEvaluator) { addHierarchicalSymbolsForDocument(symbolList: DocumentSymbol[], evaluator: TypeEvaluator) {
@ -568,51 +590,49 @@ export class SourceFile {
return; return;
} }
DocumentSymbolProvider.addHierarchicalSymbolsForDocument(symbolList, DocumentSymbolProvider.addHierarchicalSymbolsForDocument(symbolList, this._parseResults, evaluator);
this._parseResults, evaluator);
} }
addSymbolsForDocument(symbolList: SymbolInformation[], evaluator: TypeEvaluator, addSymbolsForDocument(symbolList: SymbolInformation[], evaluator: TypeEvaluator, query?: string) {
query?: string) {
// If we have no completed analysis job, there's nothing to do. // If we have no completed analysis job, there's nothing to do.
if (!this._parseResults) { if (!this._parseResults) {
return; return;
} }
DocumentSymbolProvider.addSymbolsForDocument(symbolList, query, DocumentSymbolProvider.addSymbolsForDocument(symbolList, query, this._filePath, this._parseResults, evaluator);
this._filePath, this._parseResults, evaluator);
} }
getHoverForPosition(position: Position, getHoverForPosition(position: Position, evaluator: TypeEvaluator): HoverResults | undefined {
evaluator: TypeEvaluator): HoverResults | undefined {
// If this file hasn't been bound, no hover info is available. // If this file hasn't been bound, no hover info is available.
if (this._isBindingNeeded || !this._parseResults) { if (this._isBindingNeeded || !this._parseResults) {
return undefined; return undefined;
} }
return HoverProvider.getHoverForPosition( return HoverProvider.getHoverForPosition(this._parseResults, position, evaluator);
this._parseResults, position, evaluator);
} }
getSignatureHelpForPosition(position: Position, getSignatureHelpForPosition(
importLookup: ImportLookup, evaluator: TypeEvaluator): SignatureHelpResults | undefined { position: Position,
importLookup: ImportLookup,
evaluator: TypeEvaluator
): SignatureHelpResults | undefined {
// If we have no completed analysis job, there's nothing to do. // If we have no completed analysis job, there's nothing to do.
if (!this._parseResults) { if (!this._parseResults) {
return undefined; return undefined;
} }
return SignatureHelpProvider.getSignatureHelpForPosition( return SignatureHelpProvider.getSignatureHelpForPosition(this._parseResults, position, evaluator);
this._parseResults, position, evaluator);
} }
getCompletionsForPosition(position: Position, getCompletionsForPosition(
workspacePath: string, configOptions: ConfigOptions, importResolver: ImportResolver, position: Position,
importLookup: ImportLookup, evaluator: TypeEvaluator, workspacePath: string,
moduleSymbolsCallback: () => ModuleSymbolMap): CompletionList | undefined { configOptions: ConfigOptions,
importResolver: ImportResolver,
importLookup: ImportLookup,
evaluator: TypeEvaluator,
moduleSymbolsCallback: () => ModuleSymbolMap
): CompletionList | undefined {
// If we have no completed analysis job, there's nothing to do. // If we have no completed analysis job, there's nothing to do.
if (!this._parseResults) { if (!this._parseResults) {
return undefined; return undefined;
@ -625,28 +645,46 @@ export class SourceFile {
} }
const completionProvider = new CompletionProvider( const completionProvider = new CompletionProvider(
workspacePath, this._parseResults, this._fileContents, workspacePath,
importResolver, position, this._parseResults,
this._filePath, configOptions, importLookup, evaluator, this._fileContents,
moduleSymbolsCallback); importResolver,
position,
this._filePath,
configOptions,
importLookup,
evaluator,
moduleSymbolsCallback
);
return completionProvider.getCompletionsForPosition(); return completionProvider.getCompletionsForPosition();
} }
resolveCompletionItem(configOptions: ConfigOptions, importResolver: ImportResolver, resolveCompletionItem(
importLookup: ImportLookup, evaluator: TypeEvaluator, configOptions: ConfigOptions,
moduleSymbolsCallback: () => ModuleSymbolMap, completionItem: CompletionItem) { importResolver: ImportResolver,
importLookup: ImportLookup,
evaluator: TypeEvaluator,
moduleSymbolsCallback: () => ModuleSymbolMap,
completionItem: CompletionItem
) {
if (!this._parseResults || this._fileContents === undefined) { if (!this._parseResults || this._fileContents === undefined) {
return; return;
} }
const completionData = completionItem.data as CompletionItemData; const completionData = completionItem.data as CompletionItemData;
const completionProvider = new CompletionProvider( const completionProvider = new CompletionProvider(
completionData.workspacePath, this._parseResults, this._fileContents, completionData.workspacePath,
importResolver, completionData.position, this._parseResults,
this._filePath, configOptions, importLookup, evaluator, this._fileContents,
moduleSymbolsCallback); importResolver,
completionData.position,
this._filePath,
configOptions,
importLookup,
evaluator,
moduleSymbolsCallback
);
completionProvider.resolveCompletionItem(completionItem); completionProvider.resolveCompletionItem(completionItem);
} }
@ -681,8 +719,12 @@ export class SourceFile {
timingStats.bindTime.timeOperation(() => { timingStats.bindTime.timeOperation(() => {
this._cleanParseTreeIfRequired(); this._cleanParseTreeIfRequired();
const fileInfo = this._buildFileInfo(configOptions, this._parseResults!.text, const fileInfo = this._buildFileInfo(
importLookup, builtinsScope); configOptions,
this._parseResults!.text,
importLookup,
builtinsScope
);
AnalyzerNodeInfo.setFileInfo(this._parseResults!.parseTree, fileInfo); AnalyzerNodeInfo.setFileInfo(this._parseResults!.parseTree, fileInfo);
const binder = new Binder(fileInfo); const binder = new Binder(fileInfo);
@ -702,15 +744,16 @@ export class SourceFile {
this._moduleSymbolTable = moduleScope!.symbolTable; this._moduleSymbolTable = moduleScope!.symbolTable;
}); });
} catch (e) { } catch (e) {
const message: string = (e.stack ? e.stack.toString() : undefined) || const message: string =
(e.stack ? e.stack.toString() : undefined) ||
(typeof e.message === 'string' ? e.message : undefined) || (typeof e.message === 'string' ? e.message : undefined) ||
JSON.stringify(e); JSON.stringify(e);
this._console.log( this._console.log(
`An internal error occurred while performing name binding for ${ this.getFilePath() }: ` + message); `An internal error occurred while performing name binding for ${this.getFilePath()}: ` + message
);
const diagSink = new DiagnosticSink(); const diagSink = new DiagnosticSink();
diagSink.addError(`An internal error occurred while performing name binding`, diagSink.addError(`An internal error occurred while performing name binding`, getEmptyRange());
getEmptyRange());
this._bindDiagnostics = diagSink.fetchAndClear(); this._bindDiagnostics = diagSink.fetchAndClear();
} finally { } finally {
this._isBindingInProgress = false; this._isBindingInProgress = false;
@ -739,14 +782,15 @@ export class SourceFile {
this._checkerDiagnostics = fileInfo.diagnosticSink.fetchAndClear(); this._checkerDiagnostics = fileInfo.diagnosticSink.fetchAndClear();
}); });
} catch (e) { } catch (e) {
const message: string = (e.stack ? e.stack.toString() : undefined) || const message: string =
(e.stack ? e.stack.toString() : undefined) ||
(typeof e.message === 'string' ? e.message : undefined) || (typeof e.message === 'string' ? e.message : undefined) ||
JSON.stringify(e); JSON.stringify(e);
this._console.log( this._console.log(
`An internal error occurred while while performing type checking for ${ this.getFilePath() }: ` + message); `An internal error occurred while while performing type checking for ${this.getFilePath()}: ` + message
);
const diagSink = new DiagnosticSink(); const diagSink = new DiagnosticSink();
diagSink.addError(`An internal error occurred while performing type checking`, diagSink.addError(`An internal error occurred while performing type checking`, getEmptyRange());
getEmptyRange());
// Mark the file as complete so we don't get into an infinite loop. // Mark the file as complete so we don't get into an infinite loop.
this._isCheckingNeeded = false; this._isCheckingNeeded = false;
@ -760,9 +804,12 @@ export class SourceFile {
this._diagnosticVersion++; this._diagnosticVersion++;
} }
private _buildFileInfo(configOptions: ConfigOptions, fileContents: string, private _buildFileInfo(
importLookup: ImportLookup, builtinsScope?: Scope) { configOptions: ConfigOptions,
fileContents: string,
importLookup: ImportLookup,
builtinsScope?: Scope
) {
assert(this._parseResults !== undefined); assert(this._parseResults !== undefined);
const analysisDiagnostics = new TextRangeDiagnosticSink(this._parseResults!.tokenizerOutput.lines); const analysisDiagnostics = new TextRangeDiagnosticSink(this._parseResults!.tokenizerOutput.lines);
@ -796,45 +843,43 @@ export class SourceFile {
} }
} }
private _resolveImports(importResolver: ImportResolver, moduleImports: ModuleImport[], private _resolveImports(
execEnv: ExecutionEnvironment): [ImportResult[], ImportResult?, string?, string?] { importResolver: ImportResolver,
moduleImports: ModuleImport[],
execEnv: ExecutionEnvironment
): [ImportResult[], ImportResult?, string?, string?] {
const imports: ImportResult[] = []; const imports: ImportResult[] = [];
// Always include an implicit import of the builtins module. // Always include an implicit import of the builtins module.
let builtinsImportResult: ImportResult | undefined = importResolver.resolveImport( let builtinsImportResult: ImportResult | undefined = importResolver.resolveImport(this._filePath, execEnv, {
this._filePath,
execEnv,
{
leadingDots: 0, leadingDots: 0,
nameParts: ['builtins'], nameParts: ['builtins'],
importedSymbols: undefined importedSymbols: undefined
} });
);
// Avoid importing builtins from the builtins.pyi file itself. // Avoid importing builtins from the builtins.pyi file itself.
if (builtinsImportResult.resolvedPaths.length === 0 || if (
builtinsImportResult.resolvedPaths[0] !== this.getFilePath()) { builtinsImportResult.resolvedPaths.length === 0 ||
builtinsImportResult.resolvedPaths[0] !== this.getFilePath()
) {
imports.push(builtinsImportResult); imports.push(builtinsImportResult);
} else { } else {
builtinsImportResult = undefined; builtinsImportResult = undefined;
} }
// Always include an implicit import of the typing module. // Always include an implicit import of the typing module.
const typingImportResult: ImportResult | undefined = importResolver.resolveImport( const typingImportResult: ImportResult | undefined = importResolver.resolveImport(this._filePath, execEnv, {
this._filePath,
execEnv,
{
leadingDots: 0, leadingDots: 0,
nameParts: ['typing'], nameParts: ['typing'],
importedSymbols: undefined importedSymbols: undefined
} });
);
// Avoid importing typing from the typing.pyi file itself. // Avoid importing typing from the typing.pyi file itself.
let typingModulePath: string | undefined; let typingModulePath: string | undefined;
if (typingImportResult.resolvedPaths.length === 0 || if (
typingImportResult.resolvedPaths[0] !== this.getFilePath()) { typingImportResult.resolvedPaths.length === 0 ||
typingImportResult.resolvedPaths[0] !== this.getFilePath()
) {
imports.push(typingImportResult); imports.push(typingImportResult);
typingModulePath = typingImportResult.resolvedPaths[0]; typingModulePath = typingImportResult.resolvedPaths[0];
} }
@ -842,15 +887,11 @@ export class SourceFile {
let collectionsModulePath: string | undefined; let collectionsModulePath: string | undefined;
for (const moduleImport of moduleImports) { for (const moduleImport of moduleImports) {
const importResult = importResolver.resolveImport( const importResult = importResolver.resolveImport(this._filePath, execEnv, {
this._filePath,
execEnv,
{
leadingDots: moduleImport.leadingDots, leadingDots: moduleImport.leadingDots,
nameParts: moduleImport.nameParts, nameParts: moduleImport.nameParts,
importedSymbols: moduleImport.importedSymbols importedSymbols: moduleImport.importedSymbols
} });
);
// If the file imports the stdlib 'collections' module, stash // If the file imports the stdlib 'collections' module, stash
// away its file path. The type analyzer may need this to // away its file path. The type analyzer may need this to

97
server/src/analyzer/staticExpressions.ts
View File

@ -14,9 +14,7 @@ import { KeywordType, OperatorType } from '../parser/tokenizerTypes';
// Returns undefined if the expression cannot be evaluated // Returns undefined if the expression cannot be evaluated
// statically as a bool value or true/false if it can. // statically as a bool value or true/false if it can.
export function evaluateStaticBoolExpression(node: ExpressionNode, export function evaluateStaticBoolExpression(node: ExpressionNode, execEnv: ExecutionEnvironment): boolean | undefined {
execEnv: ExecutionEnvironment): boolean | undefined {
if (node.nodeType === ParseNodeType.UnaryOperation) { if (node.nodeType === ParseNodeType.UnaryOperation) {
if (node.operator === OperatorType.Or || node.operator === OperatorType.And) { if (node.operator === OperatorType.Or || node.operator === OperatorType.And) {
const value = evaluateStaticBoolLikeExpression(node.expression, execEnv); const value = evaluateStaticBoolLikeExpression(node.expression, execEnv);
@ -41,40 +39,42 @@ export function evaluateStaticBoolExpression(node: ExpressionNode,
} }
} }
if (_isSysVersionInfoExpression(node.leftExpression) && if (_isSysVersionInfoExpression(node.leftExpression) && node.rightExpression.nodeType === ParseNodeType.Tuple) {
node.rightExpression.nodeType === ParseNodeType.Tuple) {
// Handle the special case of "sys.version_info >= (3, x)" // Handle the special case of "sys.version_info >= (3, x)"
const comparisonVersion = _convertTupleToVersion(node.rightExpression); const comparisonVersion = _convertTupleToVersion(node.rightExpression);
return _evaluateNumericBinaryOperation(node.operator, return _evaluateNumericBinaryOperation(node.operator, execEnv.pythonVersion, comparisonVersion);
execEnv.pythonVersion, comparisonVersion); } else if (
node.leftExpression.nodeType === ParseNodeType.Index &&
} else if (node.leftExpression.nodeType === ParseNodeType.Index &&
_isSysVersionInfoExpression(node.leftExpression.baseExpression) && _isSysVersionInfoExpression(node.leftExpression.baseExpression) &&
node.leftExpression.items.items.length === 1 && node.leftExpression.items.items.length === 1 &&
node.leftExpression.items.items[0].nodeType === ParseNodeType.Number && node.leftExpression.items.items[0].nodeType === ParseNodeType.Number &&
!node.leftExpression.items.items[0].isImaginary && !node.leftExpression.items.items[0].isImaginary &&
node.leftExpression.items.items[0].value === 0 && node.leftExpression.items.items[0].value === 0 &&
node.rightExpression.nodeType === ParseNodeType.Number) { node.rightExpression.nodeType === ParseNodeType.Number
) {
// Handle the special case of "sys.version_info[0] >= X" // Handle the special case of "sys.version_info[0] >= X"
return _evaluateNumericBinaryOperation(node.operator, return _evaluateNumericBinaryOperation(
Math.floor(execEnv.pythonVersion / 256), node.rightExpression.value); node.operator,
} else if (_isSysPlatformInfoExpression(node.leftExpression) && Math.floor(execEnv.pythonVersion / 256),
node.rightExpression.nodeType === ParseNodeType.StringList) { node.rightExpression.value
);
} else if (
_isSysPlatformInfoExpression(node.leftExpression) &&
node.rightExpression.nodeType === ParseNodeType.StringList
) {
// Handle the special case of "sys.platform != 'X'" // Handle the special case of "sys.platform != 'X'"
const comparisonPlatform = node.rightExpression.strings.map(s => s.value).join(''); const comparisonPlatform = node.rightExpression.strings.map(s => s.value).join('');
const expectedPlatformName = _getExpectedPlatformNameFromPlatform(execEnv); const expectedPlatformName = _getExpectedPlatformNameFromPlatform(execEnv);
return _evaluateStringBinaryOperation(node.operator, return _evaluateStringBinaryOperation(node.operator, expectedPlatformName || '', comparisonPlatform);
expectedPlatformName || '', comparisonPlatform); } else if (
} else if (_isOsNameInfoExpression(node.leftExpression) && _isOsNameInfoExpression(node.leftExpression) &&
node.rightExpression.nodeType === ParseNodeType.StringList) { node.rightExpression.nodeType === ParseNodeType.StringList
) {
// Handle the special case of "os.name == 'X'" // Handle the special case of "os.name == 'X'"
const comparisonOsName = node.rightExpression.strings.map(s => s.value).join(''); const comparisonOsName = node.rightExpression.strings.map(s => s.value).join('');
const expectedOsName = _getExpectedOsNameFromPlatform(execEnv); const expectedOsName = _getExpectedOsNameFromPlatform(execEnv);
if (expectedOsName !== undefined) { if (expectedOsName !== undefined) {
return _evaluateStringBinaryOperation(node.operator, return _evaluateStringBinaryOperation(node.operator, expectedOsName, comparisonOsName);
expectedOsName, comparisonOsName);
} }
} }
} else if (node.nodeType === ParseNodeType.Constant) { } else if (node.nodeType === ParseNodeType.Constant) {
@ -87,11 +87,12 @@ export function evaluateStaticBoolExpression(node: ExpressionNode,
if (node.value === 'TYPE_CHECKING') { if (node.value === 'TYPE_CHECKING') {
return true; return true;
} }
} else if (node.nodeType === ParseNodeType.MemberAccess && } else if (
node.nodeType === ParseNodeType.MemberAccess &&
node.memberName.value === 'TYPE_CHECKING' && node.memberName.value === 'TYPE_CHECKING' &&
node.leftExpression.nodeType === ParseNodeType.Name && node.leftExpression.nodeType === ParseNodeType.Name &&
node.leftExpression.value === 'typing') { node.leftExpression.value === 'typing'
) {
return true; return true;
} }
@ -101,9 +102,10 @@ export function evaluateStaticBoolExpression(node: ExpressionNode,
// Similar to evaluateStaticBoolExpression except that it handles // Similar to evaluateStaticBoolExpression except that it handles
// other non-bool values that are statically falsy or truthy // other non-bool values that are statically falsy or truthy
// (like "None"). // (like "None").
export function evaluateStaticBoolLikeExpression(node: ExpressionNode, export function evaluateStaticBoolLikeExpression(
execEnv: ExecutionEnvironment): boolean | undefined { node: ExpressionNode,
execEnv: ExecutionEnvironment
): boolean | undefined {
if (node.nodeType === ParseNodeType.Constant) { if (node.nodeType === ParseNodeType.Constant) {
if (node.constType === KeywordType.None) { if (node.constType === KeywordType.None) {
return false; return false;
@ -116,11 +118,12 @@ export function evaluateStaticBoolLikeExpression(node: ExpressionNode,
function _convertTupleToVersion(node: TupleNode): number | undefined { function _convertTupleToVersion(node: TupleNode): number | undefined {
let comparisonVersion: number | undefined; let comparisonVersion: number | undefined;
if (node.expressions.length === 2) { if (node.expressions.length === 2) {
if (node.expressions[0].nodeType === ParseNodeType.Number && if (
node.expressions[0].nodeType === ParseNodeType.Number &&
!node.expressions[0].isImaginary && !node.expressions[0].isImaginary &&
node.expressions[1].nodeType === ParseNodeType.Number && node.expressions[1].nodeType === ParseNodeType.Number &&
!node.expressions[1].isImaginary) { !node.expressions[1].isImaginary
) {
const majorVersion = node.expressions[0]; const majorVersion = node.expressions[0];
const minorVersion = node.expressions[1]; const minorVersion = node.expressions[1];
comparisonVersion = majorVersion.value * 256 + minorVersion.value; comparisonVersion = majorVersion.value * 256 + minorVersion.value;
@ -133,8 +136,11 @@ function _convertTupleToVersion(node: TupleNode): number | undefined {
return comparisonVersion; return comparisonVersion;
} }
function _evaluateNumericBinaryOperation(operatorType: OperatorType, leftValue: number | undefined, function _evaluateNumericBinaryOperation(
rightValue: number | undefined): any | undefined { operatorType: OperatorType,
leftValue: number | undefined,
rightValue: number | undefined
): any | undefined {
if (leftValue !== undefined && rightValue !== undefined) { if (leftValue !== undefined && rightValue !== undefined) {
if (operatorType === OperatorType.LessThan) { if (operatorType === OperatorType.LessThan) {
return leftValue < rightValue; return leftValue < rightValue;
@ -154,8 +160,11 @@ function _evaluateNumericBinaryOperation(operatorType: OperatorType, leftValue:
return undefined; return undefined;
} }
function _evaluateStringBinaryOperation(operatorType: OperatorType, function _evaluateStringBinaryOperation(
leftValue: string | undefined, rightValue: string | undefined): any | undefined { operatorType: OperatorType,
leftValue: string | undefined,
rightValue: string | undefined
): any | undefined {
if (leftValue !== undefined && rightValue !== undefined) { if (leftValue !== undefined && rightValue !== undefined) {
if (operatorType === OperatorType.Equals) { if (operatorType === OperatorType.Equals) {
return leftValue === rightValue; return leftValue === rightValue;
@ -169,9 +178,11 @@ function _evaluateStringBinaryOperation(operatorType: OperatorType,
function _isSysVersionInfoExpression(node: ExpressionNode): boolean { function _isSysVersionInfoExpression(node: ExpressionNode): boolean {
if (node.nodeType === ParseNodeType.MemberAccess) { if (node.nodeType === ParseNodeType.MemberAccess) {
if (node.leftExpression.nodeType === ParseNodeType.Name && if (
node.leftExpression.nodeType === ParseNodeType.Name &&
node.leftExpression.value === 'sys' && node.leftExpression.value === 'sys' &&
node.memberName.value === 'version_info') { node.memberName.value === 'version_info'
) {
return true; return true;
} }
} }
@ -181,9 +192,11 @@ function _isSysVersionInfoExpression(node: ExpressionNode): boolean {
function _isSysPlatformInfoExpression(node: ExpressionNode): boolean { function _isSysPlatformInfoExpression(node: ExpressionNode): boolean {
if (node.nodeType === ParseNodeType.MemberAccess) { if (node.nodeType === ParseNodeType.MemberAccess) {
if (node.leftExpression.nodeType === ParseNodeType.Name && if (
node.leftExpression.nodeType === ParseNodeType.Name &&
node.leftExpression.value === 'sys' && node.leftExpression.value === 'sys' &&
node.memberName.value === 'platform') { node.memberName.value === 'platform'
) {
return true; return true;
} }
} }
@ -193,9 +206,11 @@ function _isSysPlatformInfoExpression(node: ExpressionNode): boolean {
function _isOsNameInfoExpression(node: ExpressionNode): boolean { function _isOsNameInfoExpression(node: ExpressionNode): boolean {
if (node.nodeType === ParseNodeType.MemberAccess) { if (node.nodeType === ParseNodeType.MemberAccess) {
if (node.leftExpression.nodeType === ParseNodeType.Name && if (
node.leftExpression.nodeType === ParseNodeType.Name &&
node.leftExpression.value === 'os' && node.leftExpression.value === 'os' &&
node.memberName.value === 'name') { node.memberName.value === 'name'
) {
return true; return true;
} }
} }

13
server/src/analyzer/symbol.ts
View File

@ -129,8 +129,7 @@ export class Symbol {
// See if this node was already identified as a declaration. If so, // See if this node was already identified as a declaration. If so,
// replace it. Otherwise, add it as a new declaration to the end of // replace it. Otherwise, add it as a new declaration to the end of
// the list. // the list.
const declIndex = this._declarations.findIndex( const declIndex = this._declarations.findIndex(decl => areDeclarationsSame(decl, declaration));
decl => areDeclarationsSame(decl, declaration));
if (declIndex < 0) { if (declIndex < 0) {
this._declarations.push(declaration); this._declarations.push(declaration);
} else { } else {
@ -139,9 +138,7 @@ export class Symbol {
const curDecl = this._declarations[declIndex]; const curDecl = this._declarations[declIndex];
if (hasTypeForDeclaration(declaration)) { if (hasTypeForDeclaration(declaration)) {
this._declarations[declIndex] = declaration; this._declarations[declIndex] = declaration;
if (curDecl.type === DeclarationType.Variable && if (curDecl.type === DeclarationType.Variable && declaration.type === DeclarationType.Variable) {
declaration.type === DeclarationType.Variable) {
if (!declaration.inferredTypeSource && curDecl.inferredTypeSource) { if (!declaration.inferredTypeSource && curDecl.inferredTypeSource) {
declaration.inferredTypeSource = curDecl.inferredTypeSource; declaration.inferredTypeSource = curDecl.inferredTypeSource;
} }
@ -180,13 +177,11 @@ export class Symbol {
return true; return true;
} }
return this.getDeclarations().some( return this.getDeclarations().some(decl => hasTypeForDeclaration(decl));
decl => hasTypeForDeclaration(decl));
} }
getTypedDeclarations() { getTypedDeclarations() {
return this.getDeclarations().filter( return this.getDeclarations().filter(decl => hasTypeForDeclaration(decl));
decl => hasTypeForDeclaration(decl));
} }
getSynthesizedType() { getSynthesizedType() {

12
server/src/analyzer/symbolNameUtils.ts
View File

@ -12,16 +12,12 @@ const _underscoreOnlyRegEx = /^[_]+$/;
// Private symbol names start with a double underscore. // Private symbol names start with a double underscore.
export function isPrivateName(name: string) { export function isPrivateName(name: string) {
return name.length > 2 && return name.length > 2 && name.startsWith('__') && !name.endsWith('__');
name.startsWith('__') &&
!name.endsWith('__');
} }
// Protected symbol names start with a single underscore. // Protected symbol names start with a single underscore.
export function isProtectedName(name: string) { export function isProtectedName(name: string) {
return name.length > 1 && return name.length > 1 && name.startsWith('_') && !name.startsWith('__');
name.startsWith('_') &&
!name.startsWith('__');
} }
export function isPrivateOrProtectedName(name: string) { export function isPrivateOrProtectedName(name: string) {
@ -30,9 +26,7 @@ export function isPrivateOrProtectedName(name: string) {
// "Dunder" names start and end with two underscores. // "Dunder" names start and end with two underscores.
export function isDunderName(name: string) { export function isDunderName(name: string) {
return name.length > 4 && return name.length > 4 && name.startsWith('__') && name.endsWith('__');
name.startsWith('__') &&
name.endsWith('__');
} }
// Constants are all-caps with possible numbers and underscores. // Constants are all-caps with possible numbers and underscores.

8
server/src/analyzer/testWalker.ts
View File

@ -27,8 +27,9 @@ export class TestWalker extends ParseTreeWalker {
children.forEach(child => { children.forEach(child => {
if (child) { if (child) {
if (child.parent !== node) { if (child.parent !== node) {
fail(`Child node ${ child.nodeType } does not ` + fail(
`contain a reference to its parent ${ node.nodeType }`); `Child node ${child.nodeType} does not ` + `contain a reference to its parent ${node.nodeType}`
);
} }
} }
}); });
@ -62,8 +63,7 @@ export class TestWalker extends ParseTreeWalker {
if (!skipCheck) { if (!skipCheck) {
// Make sure the child is contained within the parent. // Make sure the child is contained within the parent.
if (child.start < node.start || TextRange.getEnd(child) > TextRange.getEnd(node)) { if (child.start < node.start || TextRange.getEnd(child) > TextRange.getEnd(node)) {
fail(`Child node ${ child.nodeType } is not ` + fail(`Child node ${child.nodeType} is not ` + `contained within its parent ${node.nodeType}`);
`contained within its parent ${ node.nodeType }`);
} }
if (prevNode) { if (prevNode) {
// Make sure the child is after the previous child. // Make sure the child is after the previous child.

3204
server/src/analyzer/typeEvaluator.ts
View File

File diff suppressed because it is too large Load Diff

122
server/src/analyzer/typeStubWriter.ts
View File

@ -9,11 +9,30 @@
*/ */
import { import {
ArgumentCategory, ArgumentNode, AssignmentNode, AugmentedAssignmentNode, ArgumentCategory,
ClassNode, DecoratorNode, ExpressionNode, ForNode, FunctionNode, IfNode, ArgumentNode,
ImportFromNode, ImportNode, ModuleNameNode, NameNode, ParameterCategory, ParameterNode, AssignmentNode,
ParseNode, ParseNodeType, StatementListNode, StringNode, TryNode, AugmentedAssignmentNode,
TypeAnnotationNode, WhileNode, WithNode ClassNode,
DecoratorNode,
ExpressionNode,
ForNode,
FunctionNode,
IfNode,
ImportFromNode,
ImportNode,
ModuleNameNode,
NameNode,
ParameterCategory,
ParameterNode,
ParseNode,
ParseNodeType,
StatementListNode,
StringNode,
TryNode,
TypeAnnotationNode,
WhileNode,
WithNode
} from '../parser/parseNodes'; } from '../parser/parseNodes';
import * as AnalyzerNodeInfo from './analyzerNodeInfo'; import * as AnalyzerNodeInfo from './analyzerNodeInfo';
import * as ParseTreeUtils from './parseTreeUtils'; import * as ParseTreeUtils from './parseTreeUtils';
@ -33,9 +52,7 @@ class TrackedImport {
} }
class TrackedImportAs extends TrackedImport { class TrackedImportAs extends TrackedImport {
constructor(importName: string, public alias: string | undefined, constructor(importName: string, public alias: string | undefined, public symbol: Symbol) {
public symbol: Symbol) {
super(importName); super(importName);
} }
} }
@ -54,9 +71,7 @@ class TrackedImportFrom extends TrackedImport {
super(importName); super(importName);
} }
addSymbol(symbol: Symbol | undefined, name: string, addSymbol(symbol: Symbol | undefined, name: string, alias: string | undefined, isAccessed = false) {
alias: string | undefined, isAccessed = false) {
if (!this.symbols.find(s => s.name === name)) { if (!this.symbols.find(s => s.name === name)) {
this.symbols.push({ this.symbols.push({
symbol, symbol,
@ -69,10 +84,7 @@ class TrackedImportFrom extends TrackedImport {
} }
class ImportSymbolWalker extends ParseTreeWalker { class ImportSymbolWalker extends ParseTreeWalker {
constructor( constructor(private _accessedImportedSymbols: Map<string, boolean>, private _treatStringsAsSymbols: boolean) {
private _accessedImportedSymbols: Map<string, boolean>,
private _treatStringsAsSymbols: boolean) {
super(); super();
} }
@ -115,8 +127,7 @@ export class TypeStubWriter extends ParseTreeWalker {
private _trackedImportFrom = new Map<string, TrackedImportFrom>(); private _trackedImportFrom = new Map<string, TrackedImportFrom>();
private _accessedImportedSymbols = new Map<string, boolean>(); private _accessedImportedSymbols = new Map<string, boolean>();
constructor(private _typingsPath: string, private _sourceFile: SourceFile, constructor(private _typingsPath: string, private _sourceFile: SourceFile, private _evaluator: TypeEvaluator) {
private _evaluator: TypeEvaluator) {
super(); super();
// As a heuristic, we'll include all of the import statements // As a heuristic, we'll include all of the import statements
@ -151,14 +162,16 @@ export class TypeStubWriter extends ParseTreeWalker {
this._emitDecorators(node.decorators); this._emitDecorators(node.decorators);
let line = `class ${className}`; let line = `class ${className}`;
if (node.arguments.length > 0) { if (node.arguments.length > 0) {
line += `(${ node.arguments.map(arg => { line += `(${node.arguments
.map(arg => {
let argString = ''; let argString = '';
if (arg.name) { if (arg.name) {
argString = arg.name.value + '='; argString = arg.name.value + '=';
} }
argString += this._printExpression(arg.valueExpression); argString += this._printExpression(arg.valueExpression);
return argString; return argString;
}).join(', ') })`; })
.join(', ')})`;
} }
line += ':'; line += ':';
this._emitLine(line); this._emitLine(line);
@ -357,14 +370,18 @@ export class TypeStubWriter extends ParseTreeWalker {
node.list.forEach(imp => { node.list.forEach(imp => {
const moduleName = this._printModuleName(imp.module); const moduleName = this._printModuleName(imp.module);
if (!this._trackedImportAs.has(moduleName)) { if (!this._trackedImportAs.has(moduleName)) {
const symbolName = imp.alias ? imp.alias.value : const symbolName = imp.alias
(imp.module.nameParts.length > 0 ? ? imp.alias.value
imp.module.nameParts[0].value : ''); : imp.module.nameParts.length > 0
? imp.module.nameParts[0].value
: '';
const symbolInfo = currentScope.lookUpSymbolRecursive(symbolName); const symbolInfo = currentScope.lookUpSymbolRecursive(symbolName);
if (symbolInfo) { if (symbolInfo) {
const trackedImportAs = new TrackedImportAs(moduleName, const trackedImportAs = new TrackedImportAs(
moduleName,
imp.alias ? imp.alias.value : undefined, imp.alias ? imp.alias.value : undefined,
symbolInfo.symbol); symbolInfo.symbol
);
this._trackedImportAs.set(moduleName, trackedImportAs); this._trackedImportAs.set(moduleName, trackedImportAs);
} }
} }
@ -385,18 +402,20 @@ export class TypeStubWriter extends ParseTreeWalker {
const moduleName = this._printModuleName(node.module); const moduleName = this._printModuleName(node.module);
let trackedImportFrom = this._trackedImportFrom.get(moduleName); let trackedImportFrom = this._trackedImportFrom.get(moduleName);
if (!trackedImportFrom) { if (!trackedImportFrom) {
trackedImportFrom = new TrackedImportFrom(moduleName, trackedImportFrom = new TrackedImportFrom(moduleName, node.isWildcardImport, node);
node.isWildcardImport, node);
this._trackedImportFrom.set(moduleName, trackedImportFrom); this._trackedImportFrom.set(moduleName, trackedImportFrom);
} }
node.imports.forEach(imp => { node.imports.forEach(imp => {
const symbolName = imp.alias ? const symbolName = imp.alias ? imp.alias.value : imp.name.value;
imp.alias.value : imp.name.value;
const symbolInfo = currentScope.lookUpSymbolRecursive(symbolName); const symbolInfo = currentScope.lookUpSymbolRecursive(symbolName);
if (symbolInfo) { if (symbolInfo) {
trackedImportFrom!.addSymbol(symbolInfo.symbol, imp.name.value, trackedImportFrom!.addSymbol(
imp.alias ? imp.alias.value : undefined, false); symbolInfo.symbol,
imp.name.value,
imp.alias ? imp.alias.value : undefined,
false
);
} }
}); });
} }
@ -445,18 +464,22 @@ export class TypeStubWriter extends ParseTreeWalker {
decorators.forEach(decorator => { decorators.forEach(decorator => {
let line = '@' + this._printExpression(decorator.leftExpression); let line = '@' + this._printExpression(decorator.leftExpression);
if (decorator.arguments) { if (decorator.arguments) {
line += `(${ decorator.arguments.map( line += `(${decorator.arguments.map(arg => this._printArgument(arg)).join(', ')})`;
arg => this._printArgument(arg)).join(', ') })`;
} }
this._emitLine(line); this._emitLine(line);
}); });
} }
private _printHeaderDocString() { private _printHeaderDocString() {
return '"""' + this._lineEnd + return (
'This type stub file was generated by pyright.' + this._lineEnd + '"""' +
'"""' + this._lineEnd + this._lineEnd +
this._lineEnd; 'This type stub file was generated by pyright.' +
this._lineEnd +
'"""' +
this._lineEnd +
this._lineEnd
);
// this._emitLine(''); // this._emitLine('');
// this._emitLine('from typing import Any, Optional'); // this._emitLine('from typing import Any, Optional');
// this._emitLine('from typing import Any, List, Dict, Optional, Tuple, Type, Union'); // this._emitLine('from typing import Any, List, Dict, Optional, Tuple, Type, Union');
@ -555,17 +578,14 @@ export class TypeStubWriter extends ParseTreeWalker {
return line + this._printExpression(node.valueExpression); return line + this._printExpression(node.valueExpression);
} }
private _printExpression(node: ExpressionNode, isType = false, private _printExpression(node: ExpressionNode, isType = false, treatStringsAsSymbols = false): string {
treatStringsAsSymbols = false): string { const importSymbolWalker = new ImportSymbolWalker(this._accessedImportedSymbols, treatStringsAsSymbols);
const importSymbolWalker = new ImportSymbolWalker(
this._accessedImportedSymbols,
treatStringsAsSymbols);
importSymbolWalker.analyze(node); importSymbolWalker.analyze(node);
return ParseTreeUtils.printExpression(node, return ParseTreeUtils.printExpression(
isType ? ParseTreeUtils.PrintExpressionFlags.ForwardDeclarations : node,
ParseTreeUtils.PrintExpressionFlags.None); isType ? ParseTreeUtils.PrintExpressionFlags.ForwardDeclarations : ParseTreeUtils.PrintExpressionFlags.None
);
} }
private _printTrackedImports() { private _printTrackedImports() {
@ -601,9 +621,9 @@ export class TypeStubWriter extends ParseTreeWalker {
lineEmitted = true; lineEmitted = true;
} }
const sortedSymbols = imp.symbols. const sortedSymbols = imp.symbols
filter(s => s.isAccessed || this._includeAllImports). .filter(s => s.isAccessed || this._includeAllImports)
sort((a, b) => { .sort((a, b) => {
if (a.name < b.name) { if (a.name < b.name) {
return -1; return -1;
} else if (a.name > b.name) { } else if (a.name > b.name) {
@ -615,13 +635,15 @@ export class TypeStubWriter extends ParseTreeWalker {
if (sortedSymbols.length > 0) { if (sortedSymbols.length > 0) {
importStr += `from ${imp.importName} import `; importStr += `from ${imp.importName} import `;
importStr += sortedSymbols.map(symbol => { importStr += sortedSymbols
.map(symbol => {
let symStr = symbol.name; let symStr = symbol.name;
if (symbol.alias) { if (symbol.alias) {
symStr += ' as ' + symbol.alias; symStr += ' as ' + symbol.alias;
} }
return symStr; return symStr;
}).join(', '); })
.join(', ');
importStr += this._lineEnd; importStr += this._lineEnd;
lineEmitted = true; lineEmitted = true;

264
server/src/analyzer/typeUtils.ts
View File

@ -12,10 +12,25 @@ import { ImportLookup } from './analyzerFileInfo';
import { DeclarationType } from './declaration'; import { DeclarationType } from './declaration';
import { Symbol, SymbolFlags, SymbolTable } from './symbol'; import { Symbol, SymbolFlags, SymbolTable } from './symbol';
import { isTypedDictMemberAccessedThroughIndex } from './symbolUtils'; import { isTypedDictMemberAccessedThroughIndex } from './symbolUtils';
import { AnyType, ClassType, combineTypes, FunctionType, isAnyOrUnknown, isNoneOrNever, import {
isTypeSame, maxTypeRecursionCount, ModuleType, NeverType, ObjectType, AnyType,
OverloadedFunctionType, SpecializedFunctionTypes, Type, TypeCategory, ClassType,
TypeVarType, UnknownType } from './types'; combineTypes,
FunctionType,
isAnyOrUnknown,
isNoneOrNever,
isTypeSame,
maxTypeRecursionCount,
ModuleType,
NeverType,
ObjectType,
OverloadedFunctionType,
SpecializedFunctionTypes,
Type,
TypeCategory,
TypeVarType,
UnknownType
} from './types';
import { TypeVarMap } from './typeVarMap'; import { TypeVarMap } from './typeVarMap';
export interface ClassMember { export interface ClassMember {
@ -89,7 +104,7 @@ export function isOptionalType(type: Type): boolean {
// Calls a callback for each subtype and combines the results // Calls a callback for each subtype and combines the results
// into a final type. // into a final type.
export function doForSubtypes(type: Type, callback: (type: Type) => (Type | undefined)): Type { export function doForSubtypes(type: Type, callback: (type: Type) => Type | undefined): Type {
if (type.category === TypeCategory.Union) { if (type.category === TypeCategory.Union) {
const newTypes: Type[] = []; const newTypes: Type[] = [];
@ -146,17 +161,16 @@ export function stripLiteralTypeArgsValue(type: Type, recursionCount = 0): Type
if (type.category === TypeCategory.Class) { if (type.category === TypeCategory.Class) {
if (type.typeArguments) { if (type.typeArguments) {
const strippedTypeArgs = type.typeArguments.map( const strippedTypeArgs = type.typeArguments.map(t =>
t => stripLiteralTypeArgsValue(stripLiteralValue(t), recursionCount + 1)); stripLiteralTypeArgsValue(stripLiteralValue(t), recursionCount + 1)
return ClassType.cloneForSpecialization(type, strippedTypeArgs, );
type.skipAbstractClassTest); return ClassType.cloneForSpecialization(type, strippedTypeArgs, type.skipAbstractClassTest);
} }
} }
if (type.category === TypeCategory.Object) { if (type.category === TypeCategory.Object) {
if (type.classType.typeArguments) { if (type.classType.typeArguments) {
type = ObjectType.create( type = ObjectType.create(stripLiteralTypeArgsValue(type.classType, recursionCount + 1) as ClassType);
stripLiteralTypeArgsValue(type.classType, recursionCount + 1) as ClassType);
} }
return type; return type;
@ -171,12 +185,12 @@ export function stripLiteralTypeArgsValue(type: Type, recursionCount = 0): Type
if (type.category === TypeCategory.Function) { if (type.category === TypeCategory.Function) {
if (type.specializedTypes) { if (type.specializedTypes) {
const strippedSpecializedTypes: SpecializedFunctionTypes = { const strippedSpecializedTypes: SpecializedFunctionTypes = {
parameterTypes: type.specializedTypes.parameterTypes.map( parameterTypes: type.specializedTypes.parameterTypes.map(t =>
t => stripLiteralTypeArgsValue(stripLiteralValue(t), recursionCount + 1)), stripLiteralTypeArgsValue(stripLiteralValue(t), recursionCount + 1)
returnType: type.specializedTypes.returnType ? ),
stripLiteralTypeArgsValue(stripLiteralValue(type.specializedTypes.returnType), returnType: type.specializedTypes.returnType
recursionCount + 1) : ? stripLiteralTypeArgsValue(stripLiteralValue(type.specializedTypes.returnType), recursionCount + 1)
undefined : undefined
}; };
type = FunctionType.cloneForSpecialization(type, strippedSpecializedTypes); type = FunctionType.cloneForSpecialization(type, strippedSpecializedTypes);
} }
@ -187,7 +201,8 @@ export function stripLiteralTypeArgsValue(type: Type, recursionCount = 0): Type
if (type.category === TypeCategory.OverloadedFunction) { if (type.category === TypeCategory.OverloadedFunction) {
const strippedOverload = OverloadedFunctionType.create(); const strippedOverload = OverloadedFunctionType.create();
strippedOverload.overloads = type.overloads.map( strippedOverload.overloads = type.overloads.map(
t => stripLiteralTypeArgsValue(t, recursionCount + 1) as FunctionType); t => stripLiteralTypeArgsValue(t, recursionCount + 1) as FunctionType
);
return strippedOverload; return strippedOverload;
} }
@ -297,8 +312,7 @@ export function isEllipsisType(type: Type): boolean {
return true; return true;
} }
return (type.category === TypeCategory.Class && return type.category === TypeCategory.Class && ClassType.isBuiltIn(type, 'ellipsis');
ClassType.isBuiltIn(type, 'ellipsis'));
} }
export function isNoReturnType(type: Type): boolean { export function isNoReturnType(type: Type): boolean {
@ -312,8 +326,7 @@ export function isNoReturnType(type: Type): boolean {
} }
export function isProperty(type: Type): boolean { export function isProperty(type: Type): boolean {
return type.category === TypeCategory.Object && return type.category === TypeCategory.Object && ClassType.isPropertyClass(type.classType);
ClassType.isPropertyClass(type.classType);
} }
// Partially specializes a type within the context of a specified // Partially specializes a type within the context of a specified
@ -335,9 +348,12 @@ export function partiallySpecializeType(type: Type, contextClassType: ClassType)
// provided or makeConcrete is true, type variables are replaced // provided or makeConcrete is true, type variables are replaced
// with a concrete type derived from the type variable if there // with a concrete type derived from the type variable if there
// is no corresponding definition in the typeVarMap. // is no corresponding definition in the typeVarMap.
export function specializeType(type: Type, typeVarMap: TypeVarMap | undefined, export function specializeType(
makeConcrete = false, recursionLevel = 0): Type { type: Type,
typeVarMap: TypeVarMap | undefined,
makeConcrete = false,
recursionLevel = 0
): Type {
// Prevent infinite recursion in case a type refers to itself. // Prevent infinite recursion in case a type refers to itself.
if (recursionLevel > 100) { if (recursionLevel > 100) {
return AnyType.create(); return AnyType.create();
@ -379,16 +395,14 @@ export function specializeType(type: Type, typeVarMap: TypeVarMap | undefined,
if (type.category === TypeCategory.Union) { if (type.category === TypeCategory.Union) {
const subtypes: Type[] = []; const subtypes: Type[] = [];
type.subtypes.forEach(typeEntry => { type.subtypes.forEach(typeEntry => {
subtypes.push(specializeType(typeEntry, typeVarMap, subtypes.push(specializeType(typeEntry, typeVarMap, makeConcrete, recursionLevel + 1));
makeConcrete, recursionLevel + 1));
}); });
return combineTypes(subtypes); return combineTypes(subtypes);
} }
if (type.category === TypeCategory.Object) { if (type.category === TypeCategory.Object) {
const classType = _specializeClassType(type.classType, const classType = _specializeClassType(type.classType, typeVarMap, makeConcrete, recursionLevel + 1);
typeVarMap, makeConcrete, recursionLevel + 1);
// Handle the "Type" special class. // Handle the "Type" special class.
if (ClassType.isBuiltIn(classType, 'Type')) { if (ClassType.isBuiltIn(classType, 'Type')) {
@ -396,8 +410,7 @@ export function specializeType(type: Type, typeVarMap: TypeVarMap | undefined,
if (typeArgs && typeArgs.length >= 1) { if (typeArgs && typeArgs.length >= 1) {
const firstTypeArg = typeArgs[0]; const firstTypeArg = typeArgs[0];
if (firstTypeArg.category === TypeCategory.Object) { if (firstTypeArg.category === TypeCategory.Object) {
return specializeType(firstTypeArg.classType, typeVarMap, return specializeType(firstTypeArg.classType, typeVarMap, makeConcrete, recursionLevel + 1);
makeConcrete, recursionLevel + 1);
} else if (firstTypeArg.category === TypeCategory.TypeVar) { } else if (firstTypeArg.category === TypeCategory.TypeVar) {
if (typeVarMap) { if (typeVarMap) {
const replacementType = typeVarMap.get(firstTypeArg.name); const replacementType = typeVarMap.get(firstTypeArg.name);
@ -418,26 +431,26 @@ export function specializeType(type: Type, typeVarMap: TypeVarMap | undefined,
} }
if (type.category === TypeCategory.Class) { if (type.category === TypeCategory.Class) {
return _specializeClassType(type, typeVarMap, return _specializeClassType(type, typeVarMap, makeConcrete, recursionLevel + 1);
makeConcrete, recursionLevel + 1);
} }
if (type.category === TypeCategory.Function) { if (type.category === TypeCategory.Function) {
return _specializeFunctionType(type, typeVarMap, return _specializeFunctionType(type, typeVarMap, makeConcrete, recursionLevel + 1);
makeConcrete, recursionLevel + 1);
} }
if (type.category === TypeCategory.OverloadedFunction) { if (type.category === TypeCategory.OverloadedFunction) {
return _specializeOverloadedFunctionType(type, typeVarMap, return _specializeOverloadedFunctionType(type, typeVarMap, makeConcrete, recursionLevel + 1);
makeConcrete, recursionLevel + 1);
} }
return type; return type;
} }
export function lookUpObjectMember(objectType: Type, memberName: string, importLookup: ImportLookup, export function lookUpObjectMember(
flags = ClassMemberLookupFlags.Default): ClassMember | undefined { objectType: Type,
memberName: string,
importLookup: ImportLookup,
flags = ClassMemberLookupFlags.Default
): ClassMember | undefined {
if (objectType.category === TypeCategory.Object) { if (objectType.category === TypeCategory.Object) {
return lookUpClassMember(objectType.classType, memberName, importLookup, flags); return lookUpClassMember(objectType.classType, memberName, importLookup, flags);
} }
@ -453,9 +466,12 @@ export function lookUpObjectMember(objectType: Type, memberName: string, importL
// ClassB[str] which inherits from Dict[_T1, int], a search for '__iter__' // ClassB[str] which inherits from Dict[_T1, int], a search for '__iter__'
// would return a class type of Dict[str, int] and a symbolType of // would return a class type of Dict[str, int] and a symbolType of
// (self) -> Iterator[str]. // (self) -> Iterator[str].
export function lookUpClassMember(classType: Type, memberName: string, importLookup: ImportLookup, export function lookUpClassMember(
flags = ClassMemberLookupFlags.Default): ClassMember | undefined { classType: Type,
memberName: string,
importLookup: ImportLookup,
flags = ClassMemberLookupFlags.Default
): ClassMember | undefined {
const declaredTypesOnly = (flags & ClassMemberLookupFlags.DeclaredTypesOnly) !== 0; const declaredTypesOnly = (flags & ClassMemberLookupFlags.DeclaredTypesOnly) !== 0;
if (classType.category === TypeCategory.Class) { if (classType.category === TypeCategory.Class) {
@ -515,7 +531,10 @@ export function lookUpClassMember(classType: Type, memberName: string, importLoo
// Recursively perform search. // Recursively perform search.
const methodType = lookUpClassMember( const methodType = lookUpClassMember(
partiallySpecializeType(baseClass, classType), partiallySpecializeType(baseClass, classType),
memberName, importLookup, flags & ~ClassMemberLookupFlags.SkipOriginalClass); memberName,
importLookup,
flags & ~ClassMemberLookupFlags.SkipOriginalClass
);
if (methodType) { if (methodType) {
return methodType; return methodType;
} }
@ -592,8 +611,7 @@ export function getTypeVarArgumentsRecursive(type: Type): TypeVarType[] {
const combinedList: TypeVarType[] = []; const combinedList: TypeVarType[] = [];
if (classType.typeArguments) { if (classType.typeArguments) {
classType.typeArguments.forEach(typeArg => { classType.typeArguments.forEach(typeArg => {
addTypeVarsToListIfUnique(combinedList, addTypeVarsToListIfUnique(combinedList, getTypeVarArgumentsRecursive(typeArg));
getTypeVarArgumentsRecursive(typeArg));
}); });
} }
@ -609,21 +627,18 @@ export function getTypeVarArgumentsRecursive(type: Type): TypeVarType[] {
} else if (type.category === TypeCategory.Union) { } else if (type.category === TypeCategory.Union) {
const combinedList: TypeVarType[] = []; const combinedList: TypeVarType[] = [];
for (const subtype of type.subtypes) { for (const subtype of type.subtypes) {
addTypeVarsToListIfUnique(combinedList, addTypeVarsToListIfUnique(combinedList, getTypeVarArgumentsRecursive(subtype));
getTypeVarArgumentsRecursive(subtype));
} }
return combinedList; return combinedList;
} else if (type.category === TypeCategory.Function) { } else if (type.category === TypeCategory.Function) {
const combinedList: TypeVarType[] = []; const combinedList: TypeVarType[] = [];
type.details.parameters.forEach(param => { type.details.parameters.forEach(param => {
addTypeVarsToListIfUnique(combinedList, addTypeVarsToListIfUnique(combinedList, getTypeVarArgumentsRecursive(param.type));
getTypeVarArgumentsRecursive(param.type));
}); });
if (type.details.declaredReturnType) { if (type.details.declaredReturnType) {
addTypeVarsToListIfUnique(combinedList, addTypeVarsToListIfUnique(combinedList, getTypeVarArgumentsRecursive(type.details.declaredReturnType));
getTypeVarArgumentsRecursive(type.details.declaredReturnType));
} }
return combinedList; return combinedList;
@ -654,9 +669,7 @@ export function stripFirstParameter(type: FunctionType): FunctionType {
// Recursively finds all of the type arguments and sets them // Recursively finds all of the type arguments and sets them
// to the specified srcType. // to the specified srcType.
export function setTypeArgumentsRecursive(destType: Type, srcType: Type, export function setTypeArgumentsRecursive(destType: Type, srcType: Type, typeVarMap: TypeVarMap, recursionCount = 0) {
typeVarMap: TypeVarMap, recursionCount = 0) {
if (typeVarMap.isLocked()) { if (typeVarMap.isLocked()) {
return; return;
} }
@ -686,16 +699,24 @@ export function setTypeArgumentsRecursive(destType: Type, srcType: Type,
setTypeArgumentsRecursive(paramType, srcType, typeVarMap, recursionCount + 1); setTypeArgumentsRecursive(paramType, srcType, typeVarMap, recursionCount + 1);
}); });
if (destType.specializedTypes.returnType) { if (destType.specializedTypes.returnType) {
setTypeArgumentsRecursive(destType.specializedTypes.returnType, srcType, setTypeArgumentsRecursive(
typeVarMap, recursionCount + 1); destType.specializedTypes.returnType,
srcType,
typeVarMap,
recursionCount + 1
);
} }
} else { } else {
destType.details.parameters.forEach(param => { destType.details.parameters.forEach(param => {
setTypeArgumentsRecursive(param.type, srcType, typeVarMap, recursionCount + 1); setTypeArgumentsRecursive(param.type, srcType, typeVarMap, recursionCount + 1);
}); });
if (destType.details.declaredReturnType) { if (destType.details.declaredReturnType) {
setTypeArgumentsRecursive(destType.details.declaredReturnType, srcType, setTypeArgumentsRecursive(
typeVarMap, recursionCount + 1); destType.details.declaredReturnType,
srcType,
typeVarMap,
recursionCount + 1
);
} }
} }
break; break;
@ -836,9 +857,11 @@ export function removeTruthinessFromType(type: Type, importLookup: ImportLookup)
// Looks up the specified symbol name within the base classes // Looks up the specified symbol name within the base classes
// of a specified class. // of a specified class.
export function getSymbolFromBaseClasses(classType: ClassType, name: string, export function getSymbolFromBaseClasses(
recursionCount = 0): SymbolWithClass | undefined { classType: ClassType,
name: string,
recursionCount = 0
): SymbolWithClass | undefined {
if (recursionCount > maxTypeRecursionCount) { if (recursionCount > maxTypeRecursionCount) {
return undefined; return undefined;
} }
@ -867,19 +890,14 @@ export function getSymbolFromBaseClasses(classType: ClassType, name: string,
} }
// Returns the declared yield type if provided, or undefined otherwise. // Returns the declared yield type if provided, or undefined otherwise.
export function getDeclaredGeneratorYieldType(functionType: FunctionType, export function getDeclaredGeneratorYieldType(functionType: FunctionType, iteratorType: Type): Type | undefined {
iteratorType: Type): Type | undefined {
const returnType = FunctionType.getSpecializedReturnType(functionType); const returnType = FunctionType.getSpecializedReturnType(functionType);
if (returnType) { if (returnType) {
const generatorTypeArgs = _getGeneratorReturnTypeArgs(returnType); const generatorTypeArgs = _getGeneratorReturnTypeArgs(returnType);
if (generatorTypeArgs && generatorTypeArgs.length >= 1 && if (generatorTypeArgs && generatorTypeArgs.length >= 1 && iteratorType.category === TypeCategory.Class) {
iteratorType.category === TypeCategory.Class) {
// The yield type is the first type arg. Wrap it in an iterator. // The yield type is the first type arg. Wrap it in an iterator.
return ObjectType.create(ClassType.cloneForSpecialization( return ObjectType.create(ClassType.cloneForSpecialization(iteratorType, [generatorTypeArgs[0]]));
iteratorType, [generatorTypeArgs[0]]));
} }
// If the return type isn't a Generator, assume that it's the // If the return type isn't a Generator, assume that it's the
@ -936,9 +954,7 @@ export function convertClassToObject(type: Type): Type {
}); });
} }
export function getMembersForClass(classType: ClassType, symbolTable: SymbolTable, export function getMembersForClass(classType: ClassType, symbolTable: SymbolTable, includeInstanceVars: boolean) {
includeInstanceVars: boolean) {
_getMembersForClassRecursive(classType, symbolTable, includeInstanceVars); _getMembersForClassRecursive(classType, symbolTable, includeInstanceVars);
} }
@ -957,9 +973,7 @@ export function getMembersForModule(moduleType: ModuleType, symbolTable: SymbolT
}); });
} }
export function containsUnknown(type: Type, allowUnknownTypeArgsForClasses = false, export function containsUnknown(type: Type, allowUnknownTypeArgsForClasses = false, recursionCount = 0): boolean {
recursionCount = 0): boolean {
if (recursionCount > maxTypeRecursionCount) { if (recursionCount > maxTypeRecursionCount) {
return false; return false;
} }
@ -971,9 +985,7 @@ export function containsUnknown(type: Type, allowUnknownTypeArgsForClasses = fal
// See if a union contains an unknown type. // See if a union contains an unknown type.
if (type.category === TypeCategory.Union) { if (type.category === TypeCategory.Union) {
for (const subtype of type.subtypes) { for (const subtype of type.subtypes) {
if (containsUnknown(subtype, allowUnknownTypeArgsForClasses, if (containsUnknown(subtype, allowUnknownTypeArgsForClasses, recursionCount + 1)) {
recursionCount + 1)) {
return true; return true;
} }
} }
@ -989,9 +1001,7 @@ export function containsUnknown(type: Type, allowUnknownTypeArgsForClasses = fal
if (type.category === TypeCategory.Class) { if (type.category === TypeCategory.Class) {
if (type.typeArguments && !allowUnknownTypeArgsForClasses) { if (type.typeArguments && !allowUnknownTypeArgsForClasses) {
for (const argType of type.typeArguments) { for (const argType of type.typeArguments) {
if (containsUnknown(argType, allowUnknownTypeArgsForClasses, if (containsUnknown(argType, allowUnknownTypeArgsForClasses, recursionCount + 1)) {
recursionCount + 1)) {
return true; return true;
} }
} }
@ -1024,18 +1034,19 @@ export function containsUnknown(type: Type, allowUnknownTypeArgsForClasses = fal
return false; return false;
} }
function _getMembersForClassRecursive(classType: ClassType, function _getMembersForClassRecursive(
symbolTable: SymbolTable, includeInstanceVars: boolean, classType: ClassType,
recursionCount = 0) { symbolTable: SymbolTable,
includeInstanceVars: boolean,
recursionCount = 0
) {
if (recursionCount > maxTypeRecursionCount) { if (recursionCount > maxTypeRecursionCount) {
return; return;
} }
classType.details.baseClasses.forEach(baseClassType => { classType.details.baseClasses.forEach(baseClassType => {
if (baseClassType.category === TypeCategory.Class) { if (baseClassType.category === TypeCategory.Class) {
_getMembersForClassRecursive(baseClassType, _getMembersForClassRecursive(baseClassType, symbolTable, includeInstanceVars, recursionCount + 1);
symbolTable, includeInstanceVars, recursionCount + 1);
} }
}); });
@ -1052,9 +1063,12 @@ function _getMembersForClassRecursive(classType: ClassType,
}); });
} }
function _specializeClassType(classType: ClassType, typeVarMap: TypeVarMap | undefined, function _specializeClassType(
makeConcrete: boolean, recursionLevel: number): ClassType { classType: ClassType,
typeVarMap: TypeVarMap | undefined,
makeConcrete: boolean,
recursionLevel: number
): ClassType {
// Handle the common case where the class has no type parameters. // Handle the common case where the class has no type parameters.
if (ClassType.getTypeParameters(classType).length === 0) { if (ClassType.getTypeParameters(classType).length === 0) {
return classType; return classType;
@ -1066,8 +1080,7 @@ function _specializeClassType(classType: ClassType, typeVarMap: TypeVarMap | und
// If type args were previously provided, specialize them. // If type args were previously provided, specialize them.
if (classType.typeArguments) { if (classType.typeArguments) {
newTypeArgs = classType.typeArguments.map(oldTypeArgType => { newTypeArgs = classType.typeArguments.map(oldTypeArgType => {
const newTypeArgType = specializeType(oldTypeArgType, const newTypeArgType = specializeType(oldTypeArgType, typeVarMap, makeConcrete, recursionLevel + 1);
typeVarMap, makeConcrete, recursionLevel + 1);
if (newTypeArgType !== oldTypeArgType) { if (newTypeArgType !== oldTypeArgType) {
specializationNeeded = true; specializationNeeded = true;
} }
@ -1118,13 +1131,16 @@ export function getConcreteTypeFromTypeVar(type: TypeVarType, recursionLevel = 0
return UnknownType.create(); return UnknownType.create();
} }
function _specializeOverloadedFunctionType(type: OverloadedFunctionType, function _specializeOverloadedFunctionType(
typeVarMap: TypeVarMap | undefined, makeConcrete: boolean, type: OverloadedFunctionType,
recursionLevel: number): OverloadedFunctionType { typeVarMap: TypeVarMap | undefined,
makeConcrete: boolean,
recursionLevel: number
): OverloadedFunctionType {
// Specialize each of the functions in the overload. // Specialize each of the functions in the overload.
const overloads = type.overloads.map( const overloads = type.overloads.map(entry =>
entry => _specializeFunctionType(entry, typeVarMap, makeConcrete, recursionLevel)); _specializeFunctionType(entry, typeVarMap, makeConcrete, recursionLevel)
);
// Construct a new overload with the specialized function types. // Construct a new overload with the specialized function types.
const newOverloadType = OverloadedFunctionType.create(); const newOverloadType = OverloadedFunctionType.create();
@ -1135,15 +1151,19 @@ function _specializeOverloadedFunctionType(type: OverloadedFunctionType,
return newOverloadType; return newOverloadType;
} }
function _specializeFunctionType(functionType: FunctionType, function _specializeFunctionType(
typeVarMap: TypeVarMap | undefined, makeConcrete: boolean, functionType: FunctionType,
recursionLevel: number): FunctionType { typeVarMap: TypeVarMap | undefined,
makeConcrete: boolean,
const declaredReturnType = functionType.specializedTypes && functionType.specializedTypes.returnType ? recursionLevel: number
functionType.specializedTypes.returnType : functionType.details.declaredReturnType; ): FunctionType {
const specializedReturnType = declaredReturnType ? const declaredReturnType =
specializeType(declaredReturnType, typeVarMap, makeConcrete, recursionLevel + 1) : functionType.specializedTypes && functionType.specializedTypes.returnType
undefined; ? functionType.specializedTypes.returnType
: functionType.details.declaredReturnType;
const specializedReturnType = declaredReturnType
? specializeType(declaredReturnType, typeVarMap, makeConcrete, recursionLevel + 1)
: undefined;
let typesRequiredSpecialization = declaredReturnType !== specializedReturnType; let typesRequiredSpecialization = declaredReturnType !== specializedReturnType;
const specializedParameters: SpecializedFunctionTypes = { const specializedParameters: SpecializedFunctionTypes = {
@ -1153,8 +1173,7 @@ function _specializeFunctionType(functionType: FunctionType,
for (let i = 0; i < functionType.details.parameters.length; i++) { for (let i = 0; i < functionType.details.parameters.length; i++) {
const paramType = FunctionType.getEffectiveParameterType(functionType, i); const paramType = FunctionType.getEffectiveParameterType(functionType, i);
const specializedType = specializeType(paramType, const specializedType = specializeType(paramType, typeVarMap, makeConcrete, recursionLevel + 1);
typeVarMap, makeConcrete, recursionLevel + 1);
specializedParameters.parameterTypes.push(specializedType); specializedParameters.parameterTypes.push(specializedType);
if (paramType !== specializedType) { if (paramType !== specializedType) {
@ -1197,9 +1216,10 @@ export function requiresSpecialization(type: Type, recursionCount = 0): boolean
return false; return false;
} }
return type.typeArguments.find( return (
typeArg => requiresSpecialization(typeArg, recursionCount + 1) type.typeArguments.find(typeArg => requiresSpecialization(typeArg, recursionCount + 1)) !==
) !== undefined; undefined
);
} }
// If there are any type parameters, we need to specialize // If there are any type parameters, we need to specialize
@ -1226,8 +1246,10 @@ export function requiresSpecialization(type: Type, recursionCount = 0): boolean
} }
} }
const declaredReturnType = type.specializedTypes && type.specializedTypes.returnType ? const declaredReturnType =
type.specializedTypes.returnType : type.details.declaredReturnType; type.specializedTypes && type.specializedTypes.returnType
? type.specializedTypes.returnType
: type.details.declaredReturnType;
if (declaredReturnType) { if (declaredReturnType) {
if (requiresSpecialization(declaredReturnType, recursionCount + 1)) { if (requiresSpecialization(declaredReturnType, recursionCount + 1)) {
return true; return true;
@ -1238,13 +1260,11 @@ export function requiresSpecialization(type: Type, recursionCount = 0): boolean
} }
case TypeCategory.OverloadedFunction: { case TypeCategory.OverloadedFunction: {
return type.overloads.find( return type.overloads.find(overload => requiresSpecialization(overload, recursionCount + 1)) !== undefined;
overload => requiresSpecialization(overload, recursionCount + 1)) !== undefined;
} }
case TypeCategory.Union: { case TypeCategory.Union: {
return type.subtypes.find( return type.subtypes.find(type => requiresSpecialization(type, recursionCount + 1)) !== undefined;
type => requiresSpecialization(type, recursionCount + 1)) !== undefined;
} }
case TypeCategory.TypeVar: { case TypeCategory.TypeVar: {
@ -1262,10 +1282,10 @@ export function printLiteralValue(type: ObjectType): string {
} }
let literalStr: string; let literalStr: string;
if (typeof(literalValue) === 'string') { if (typeof literalValue === 'string') {
const prefix = (type.classType.details.name === 'bytes') ? 'b' : ''; const prefix = type.classType.details.name === 'bytes' ? 'b' : '';
literalStr = `${prefix}'${literalValue.toString()}'`; literalStr = `${prefix}'${literalValue.toString()}'`;
} else if (typeof(literalValue) === 'boolean') { } else if (typeof literalValue === 'boolean') {
literalStr = literalValue ? 'True' : 'False'; literalStr = literalValue ? 'True' : 'False';
} else { } else {
literalStr = literalValue.toString(); literalStr = literalValue.toString();

4
server/src/analyzer/typeVarMap.ts
View File

@ -9,8 +9,8 @@
* evaluator to "solve" for the type of each type variable. * evaluator to "solve" for the type of each type variable.
*/ */
import { assert } from "../common/debug"; import { assert } from '../common/debug';
import { Type } from "./types"; import { Type } from './types';
export class TypeVarMap { export class TypeVarMap {
private _typeMap: Map<string, Type>; private _typeMap: Map<string, Type>;

120
server/src/analyzer/types.ts
View File

@ -56,9 +56,19 @@ export const enum TypeCategory {
TypeVar TypeVar
} }
export type Type = UnboundType | UnknownType | AnyType | NoneType | NeverType | export type Type =
FunctionType | OverloadedFunctionType | ClassType | | UnboundType
ObjectType | ModuleType | UnionType | TypeVarType; | UnknownType
| AnyType
| NoneType
| NeverType
| FunctionType
| OverloadedFunctionType
| ClassType
| ObjectType
| ModuleType
| UnionType
| TypeVarType;
export type LiteralValue = number | boolean | string; export type LiteralValue = number | boolean | string;
@ -224,9 +234,7 @@ export interface ClassType extends TypeBase {
} }
export namespace ClassType { export namespace ClassType {
export function create(name: string, flags: ClassTypeFlags, export function create(name: string, flags: ClassTypeFlags, typeSourceId: TypeSourceId, docString?: string) {
typeSourceId: TypeSourceId, docString?: string) {
const newClass: ClassType = { const newClass: ClassType = {
category: TypeCategory.Class, category: TypeCategory.Class,
details: { details: {
@ -244,11 +252,12 @@ export namespace ClassType {
return newClass; return newClass;
} }
export function cloneForSpecialization(classType: ClassType, export function cloneForSpecialization(
typeArguments: Type[] | undefined, skipAbstractClassTest = false): ClassType { classType: ClassType,
typeArguments: Type[] | undefined,
const newClassType = create(classType.details.name, skipAbstractClassTest = false
classType.details.flags, classType.details.typeSourceId); ): ClassType {
const newClassType = create(classType.details.name, classType.details.flags, classType.details.typeSourceId);
newClassType.details = classType.details; newClassType.details = classType.details;
newClassType.typeArguments = typeArguments; newClassType.typeArguments = typeArguments;
if (skipAbstractClassTest) { if (skipAbstractClassTest) {
@ -260,8 +269,7 @@ export namespace ClassType {
// Specifies whether the class type is generic (unspecialized) // Specifies whether the class type is generic (unspecialized)
// or specialized. // or specialized.
export function isGeneric(classType: ClassType) { export function isGeneric(classType: ClassType) {
return classType.details.typeParameters.length > 0 && return classType.details.typeParameters.length > 0 && classType.typeArguments === undefined;
classType.typeArguments === undefined;
} }
export function isSpecialBuiltIn(classType: ClassType, className?: string) { export function isSpecialBuiltIn(classType: ClassType, className?: string) {
@ -290,19 +298,20 @@ export namespace ClassType {
export function isProtocol(classType: ClassType) { export function isProtocol(classType: ClassType) {
// Does the class directly 'derive' from "Protocol"? // Does the class directly 'derive' from "Protocol"?
return classType.details.baseClasses.find(baseClass => { return (
classType.details.baseClasses.find(baseClass => {
if (baseClass.category === TypeCategory.Class) { if (baseClass.category === TypeCategory.Class) {
if (isBuiltIn(baseClass, 'Protocol')) { if (isBuiltIn(baseClass, 'Protocol')) {
return true; return true;
} }
} }
return false; return false;
}) !== undefined; }) !== undefined
);
} }
export function hasAbstractMethods(classType: ClassType) { export function hasAbstractMethods(classType: ClassType) {
return !!(classType.details.flags & ClassTypeFlags.HasAbstractMethods) && return !!(classType.details.flags & ClassTypeFlags.HasAbstractMethods) && !classType.skipAbstractClassTest;
!classType.skipAbstractClassTest;
} }
export function supportsAbstractMethods(classType: ClassType) { export function supportsAbstractMethods(classType: ClassType) {
@ -367,10 +376,8 @@ export namespace ClassType {
// If either or both have aliases (e.g. List -> list), use the // If either or both have aliases (e.g. List -> list), use the
// aliases for comparison purposes. // aliases for comparison purposes.
const class1Details = classType.details.aliasClass ? const class1Details = classType.details.aliasClass ? classType.details.aliasClass.details : classType.details;
classType.details.aliasClass.details : classType.details; const class2Details = type2.details.aliasClass ? type2.details.aliasClass.details : type2.details;
const class2Details = type2.details.aliasClass ?
type2.details.aliasClass.details : type2.details;
if (class1Details === class2Details) { if (class1Details === class2Details) {
return true; return true;
@ -378,34 +385,34 @@ export namespace ClassType {
// Compare most of the details fields. We intentionally skip the isAbstractClass // Compare most of the details fields. We intentionally skip the isAbstractClass
// flag because it gets set dynamically. // flag because it gets set dynamically.
if (class1Details.name !== class2Details.name || if (
class1Details.name !== class2Details.name ||
class1Details.flags !== class2Details.flags || class1Details.flags !== class2Details.flags ||
class1Details.typeSourceId !== class2Details.typeSourceId || class1Details.typeSourceId !== class2Details.typeSourceId ||
class1Details.baseClasses.length !== class2Details.baseClasses.length || class1Details.baseClasses.length !== class2Details.baseClasses.length ||
class1Details.typeParameters.length !== class2Details.typeParameters.length) { class1Details.typeParameters.length !== class2Details.typeParameters.length
) {
return false; return false;
} }
for (let i = 0; i < class1Details.baseClasses.length; i++) { for (let i = 0; i < class1Details.baseClasses.length; i++) {
if (!isTypeSame(class1Details.baseClasses[i], class2Details.baseClasses[i], if (!isTypeSame(class1Details.baseClasses[i], class2Details.baseClasses[i], recursionCount + 1)) {
recursionCount + 1)) {
return false; return false;
} }
} }
if (class1Details.metaClass || class2Details.metaClass) { if (class1Details.metaClass || class2Details.metaClass) {
if (!class1Details.metaClass || !class2Details.metaClass || if (
!isTypeSame(class1Details.metaClass, class2Details.metaClass)) { !class1Details.metaClass ||
!class2Details.metaClass ||
!isTypeSame(class1Details.metaClass, class2Details.metaClass)
) {
return false; return false;
} }
} }
for (let i = 0; i < class1Details.typeParameters.length; i++) { for (let i = 0; i < class1Details.typeParameters.length; i++) {
if (!isTypeSame(class1Details.typeParameters[i], class2Details.typeParameters[i], if (!isTypeSame(class1Details.typeParameters[i], class2Details.typeParameters[i], recursionCount + 1)) {
recursionCount + 1)) {
return false; return false;
} }
} }
@ -417,10 +424,11 @@ export namespace ClassType {
} }
for (let i = 0; i < dataClassEntries1.length; i++) { for (let i = 0; i < dataClassEntries1.length; i++) {
if (dataClassEntries1[i].name !== dataClassEntries2[i].name || if (
dataClassEntries1[i].name !== dataClassEntries2[i].name ||
dataClassEntries1[i].hasDefault !== dataClassEntries2[i].hasDefault || dataClassEntries1[i].hasDefault !== dataClassEntries2[i].hasDefault ||
!isTypeSame(dataClassEntries1[i].type, dataClassEntries2[i].type, recursionCount + 1)) { !isTypeSame(dataClassEntries1[i].type, dataClassEntries2[i].type, recursionCount + 1)
) {
return false; return false;
} }
} }
@ -460,9 +468,11 @@ export namespace ClassType {
// array to inheritanceChain, it will be filled in by // array to inheritanceChain, it will be filled in by
// the call to include the chain of inherited classes starting // the call to include the chain of inherited classes starting
// with type2 and ending with this type. // with type2 and ending with this type.
export function isDerivedFrom(subclassType: ClassType, export function isDerivedFrom(
parentClassType: ClassType, inheritanceChain?: InheritanceChain): boolean { subclassType: ClassType,
parentClassType: ClassType,
inheritanceChain?: InheritanceChain
): boolean {
// Is it the exact same class? // Is it the exact same class?
if (isSameGenericClass(subclassType, parentClassType)) { if (isSameGenericClass(subclassType, parentClassType)) {
if (inheritanceChain) { if (inheritanceChain) {
@ -650,8 +660,7 @@ export namespace FunctionType {
// If we strip off the first parameter, this is no longer an // If we strip off the first parameter, this is no longer an
// instance method or class method. // instance method or class method.
if (deleteFirstParam) { if (deleteFirstParam) {
newFunction.details.flags &= ~(FunctionTypeFlags.ConstructorMethod | newFunction.details.flags &= ~(FunctionTypeFlags.ConstructorMethod | FunctionTypeFlags.ClassMethod);
FunctionTypeFlags.ClassMethod);
newFunction.details.flags |= FunctionTypeFlags.StaticMethod; newFunction.details.flags |= FunctionTypeFlags.StaticMethod;
newFunction.ignoreFirstParamOfDeclaration = true; newFunction.ignoreFirstParamOfDeclaration = true;
} }
@ -671,9 +680,10 @@ export namespace FunctionType {
// Creates a shallow copy of the function type with new // Creates a shallow copy of the function type with new
// specialized types. The clone shares the _functionDetails // specialized types. The clone shares the _functionDetails
// with the object being cloned. // with the object being cloned.
export function cloneForSpecialization(type: FunctionType, export function cloneForSpecialization(
specializedTypes: SpecializedFunctionTypes): FunctionType { type: FunctionType,
specializedTypes: SpecializedFunctionTypes
): FunctionType {
const newFunction = create(type.details.flags, type.details.docString); const newFunction = create(type.details.flags, type.details.docString);
newFunction.details = type.details; newFunction.details = type.details;
@ -684,8 +694,13 @@ export namespace FunctionType {
} }
export function isInstanceMethod(type: FunctionType): boolean { export function isInstanceMethod(type: FunctionType): boolean {
return (type.details.flags & (FunctionTypeFlags.ConstructorMethod | return (
FunctionTypeFlags.StaticMethod | FunctionTypeFlags.ClassMethod)) === 0; (type.details.flags &
(FunctionTypeFlags.ConstructorMethod |
FunctionTypeFlags.StaticMethod |
FunctionTypeFlags.ClassMethod)) ===
0
);
} }
export function isConstructorMethod(type: FunctionType): boolean { export function isConstructorMethod(type: FunctionType): boolean {
@ -758,8 +773,9 @@ export namespace FunctionType {
} }
export function getSpecializedReturnType(type: FunctionType) { export function getSpecializedReturnType(type: FunctionType) {
return type.specializedTypes && type.specializedTypes.returnType ? return type.specializedTypes && type.specializedTypes.returnType
type.specializedTypes.returnType : type.details.declaredReturnType; ? type.specializedTypes.returnType
: type.details.declaredReturnType;
} }
} }
@ -886,13 +902,11 @@ export namespace TypeVarType {
} }
export function isNoneOrNever(type: Type): boolean { export function isNoneOrNever(type: Type): boolean {
return type.category === TypeCategory.None || return type.category === TypeCategory.None || type.category === TypeCategory.Never;
type.category === TypeCategory.Never;
} }
export function isAnyOrUnknown(type: Type): boolean { export function isAnyOrUnknown(type: Type): boolean {
if (type.category === TypeCategory.Any || if (type.category === TypeCategory.Any || type.category === TypeCategory.Unknown) {
type.category === TypeCategory.Unknown) {
return true; return true;
} }
@ -1005,9 +1019,7 @@ export function isTypeSame(type1: Type, type2: Type, recursionCount = 0): boolea
return2Type = functionType2.specializedTypes.returnType; return2Type = functionType2.specializedTypes.returnType;
} }
if (return1Type || return2Type) { if (return1Type || return2Type) {
if (!return1Type || !return2Type || if (!return1Type || !return2Type || !isTypeSame(return1Type, return2Type, recursionCount + 1)) {
!isTypeSame(return1Type, return2Type, recursionCount + 1)) {
return false; return false;
} }
} }

67
server/src/common/collectionUtils.ts
View File

@ -11,7 +11,11 @@ import { compareValues, Comparison, equateValues, isArray } from './core';
export const emptyArray: never[] = [] as never[]; export const emptyArray: never[] = [] as never[];
export type EqualityComparer<T> = (a: T, b: T) => boolean; export type EqualityComparer<T> = (a: T, b: T) => boolean;
export function contains<T>(array: readonly T[] | undefined, value: T, equalityComparer: EqualityComparer<T> = equateValues): boolean { export function contains<T>(
array: readonly T[] | undefined,
value: T,
equalityComparer: EqualityComparer<T> = equateValues
): boolean {
if (array) { if (array) {
for (const v of array) { for (const v of array) {
if (equalityComparer(v, value)) { if (equalityComparer(v, value)) {
@ -36,20 +40,29 @@ export interface Push<T> {
* appended. * appended.
*/ */
export function append<TArray extends any[] | undefined, TValue extends NonNullable<TArray>[number] | undefined>( export function append<TArray extends any[] | undefined, TValue extends NonNullable<TArray>[number] | undefined>(
to: TArray, value: TValue): [undefined, undefined] extends [TArray, TValue] ? TArray : NonNullable<TArray>[number][]; to: TArray,
value: TValue
): [undefined, undefined] extends [TArray, TValue] ? TArray : NonNullable<TArray>[number][];
export function append<T>(to: T[], value: T | undefined): T[]; export function append<T>(to: T[], value: T | undefined): T[];
export function append<T>(to: T[] | undefined, value: T): T[]; export function append<T>(to: T[] | undefined, value: T): T[];
export function append<T>(to: T[] | undefined, value: T | undefined): T[] | undefined; export function append<T>(to: T[] | undefined, value: T | undefined): T[] | undefined;
export function append<T>(to: Push<T>, value: T | undefined): void; export function append<T>(to: Push<T>, value: T | undefined): void;
export function append<T>(to: T[], value: T | undefined): T[] | undefined { export function append<T>(to: T[], value: T | undefined): T[] | undefined {
if (value === undefined) { return to; } if (value === undefined) {
if (to === undefined) { return [value]; } return to;
}
if (to === undefined) {
return [value];
}
to.push(value); to.push(value);
return to; return to;
} }
/** Works like Array.prototype.find, returning `undefined` if no element satisfying the predicate is found. */ /** Works like Array.prototype.find, returning `undefined` if no element satisfying the predicate is found. */
export function find<T, U extends T>(array: readonly T[], predicate: (element: T, index: number) => element is U): U | undefined; export function find<T, U extends T>(
array: readonly T[],
predicate: (element: T, index: number) => element is U
): U | undefined;
export function find<T>(array: readonly T[], predicate: (element: T, index: number) => boolean): T | undefined; export function find<T>(array: readonly T[], predicate: (element: T, index: number) => boolean): T | undefined;
export function find<T>(array: readonly T[], predicate: (element: T, index: number) => boolean): T | undefined { export function find<T>(array: readonly T[], predicate: (element: T, index: number) => boolean): T | undefined {
for (let i = 0; i < array.length; i++) { for (let i = 0; i < array.length; i++) {
@ -80,10 +93,24 @@ function toOffset(array: readonly any[], offset: number) {
* @param end The offset in `from` at which to stop copying values (non-inclusive). * @param end The offset in `from` at which to stop copying values (non-inclusive).
*/ */
export function addRange<T>(to: T[], from: readonly T[] | undefined, start?: number, end?: number): T[]; export function addRange<T>(to: T[], from: readonly T[] | undefined, start?: number, end?: number): T[];
export function addRange<T>(to: T[] | undefined, from: readonly T[] | undefined, start?: number, end?: number): T[] | undefined; export function addRange<T>(
export function addRange<T>(to: T[] | undefined, from: readonly T[] | undefined, start?: number, end?: number): T[] | undefined { to: T[] | undefined,
if (from === undefined || from.length === 0) { return to; } from: readonly T[] | undefined,
if (to === undefined) { return from.slice(start, end); } start?: number,
end?: number
): T[] | undefined;
export function addRange<T>(
to: T[] | undefined,
from: readonly T[] | undefined,
start?: number,
end?: number
): T[] | undefined {
if (from === undefined || from.length === 0) {
return to;
}
if (to === undefined) {
return from.slice(start, end);
}
start = start === undefined ? 0 : toOffset(from, start); start = start === undefined ? 0 : toOffset(from, start);
end = end === undefined ? from.length : toOffset(from, end); end = end === undefined ? from.length : toOffset(from, end);
for (let i = start; i < end && i < from.length; i++) { for (let i = start; i < end && i < from.length; i++) {
@ -139,7 +166,7 @@ function indicesOf(array: readonly unknown[]): number[] {
export function stableSort<T>(array: readonly T[], comparer: Comparer<T>): SortedReadonlyArray<T> { export function stableSort<T>(array: readonly T[], comparer: Comparer<T>): SortedReadonlyArray<T> {
const indices = indicesOf(array); const indices = indicesOf(array);
stableSortIndices(array, indices, comparer); stableSortIndices(array, indices, comparer);
return indices.map(i => array[i]) as SortedArray<T> as SortedReadonlyArray<T>; return (indices.map(i => array[i]) as SortedArray<T>) as SortedReadonlyArray<T>;
} }
function stableSortIndices<T>(array: readonly T[], indices: number[], comparer: Comparer<T>) { function stableSortIndices<T>(array: readonly T[], indices: number[], comparer: Comparer<T>) {
@ -193,9 +220,13 @@ export function every<T>(array: readonly T[], callback: (element: T, index: numb
* @param keyComparer A callback used to compare two keys in a sorted array. * @param keyComparer A callback used to compare two keys in a sorted array.
* @param offset An offset into `array` at which to start the search. * @param offset An offset into `array` at which to start the search.
*/ */
export function binarySearch<T, U>(array: readonly T[], value: T, export function binarySearch<T, U>(
keySelector: (v: T) => U, keyComparer: Comparer<U>, offset?: number): number { array: readonly T[],
value: T,
keySelector: (v: T) => U,
keyComparer: Comparer<U>,
offset?: number
): number {
return binarySearchKey(array, keySelector(value), keySelector, keyComparer, offset); return binarySearchKey(array, keySelector(value), keySelector, keyComparer, offset);
} }
@ -209,9 +240,13 @@ export function binarySearch<T, U>(array: readonly T[], value: T,
* @param keyComparer A callback used to compare two keys in a sorted array. * @param keyComparer A callback used to compare two keys in a sorted array.
* @param offset An offset into `array` at which to start the search. * @param offset An offset into `array` at which to start the search.
*/ */
export function binarySearchKey<T, U>(array: readonly T[], key: U, export function binarySearchKey<T, U>(
keySelector: (v: T) => U, keyComparer: Comparer<U>, offset?: number): number { array: readonly T[],
key: U,
keySelector: (v: T) => U,
keyComparer: Comparer<U>,
offset?: number
): number {
if (!some(array)) { if (!some(array)) {
return -1; return -1;
} }

271
server/src/common/configOptions.ts
View File

@ -11,8 +11,7 @@ import { isAbsolute } from 'path';
import { ConsoleInterface } from './console'; import { ConsoleInterface } from './console';
import { DiagnosticRule } from './diagnosticRules'; import { DiagnosticRule } from './diagnosticRules';
import { combinePaths, ensureTrailingDirectorySeparator, FileSpec, import { combinePaths, ensureTrailingDirectorySeparator, FileSpec, getFileSpec, normalizePath } from './pathUtils';
getFileSpec, normalizePath } from './pathUtils';
import { latestStablePythonVersion, PythonVersion, versionFromString } from './pythonVersion'; import { latestStablePythonVersion, PythonVersion, versionFromString } from './pythonVersion';
export class ExecutionEnvironment { export class ExecutionEnvironment {
@ -163,9 +162,7 @@ export interface DiagnosticSettings {
reportImplicitStringConcatenation: DiagnosticLevel; reportImplicitStringConcatenation: DiagnosticLevel;
} }
export function cloneDiagnosticSettings( export function cloneDiagnosticSettings(diagSettings: DiagnosticSettings): DiagnosticSettings {
diagSettings: DiagnosticSettings): DiagnosticSettings {
// Create a shallow copy of the existing object. // Create a shallow copy of the existing object.
return Object.assign({}, diagSettings); return Object.assign({}, diagSettings);
} }
@ -399,14 +396,12 @@ export class ConfigOptions {
// execution environment is used. // execution environment is used.
findExecEnvironment(filePath: string): ExecutionEnvironment { findExecEnvironment(filePath: string): ExecutionEnvironment {
let execEnv = this.executionEnvironments.find(env => { let execEnv = this.executionEnvironments.find(env => {
const envRoot = ensureTrailingDirectorySeparator( const envRoot = ensureTrailingDirectorySeparator(normalizePath(combinePaths(this.projectRoot, env.root)));
normalizePath(combinePaths(this.projectRoot, env.root)));
return filePath.startsWith(envRoot); return filePath.startsWith(envRoot);
}); });
if (!execEnv) { if (!execEnv) {
execEnv = new ExecutionEnvironment(this.projectRoot, execEnv = new ExecutionEnvironment(this.projectRoot, this.defaultPythonVersion, this.defaultPythonPlatform);
this.defaultPythonVersion, this.defaultPythonPlatform);
} }
return execEnv; return execEnv;
@ -495,194 +490,270 @@ export class ConfigOptions {
this.diagnosticSettings = { this.diagnosticSettings = {
// Use strict inference rules for list expressions? // Use strict inference rules for list expressions?
strictListInference: this._convertBoolean( strictListInference: this._convertBoolean(
configObj.strictListInference, DiagnosticRule.strictListInference, configObj.strictListInference,
defaultSettings.strictListInference), DiagnosticRule.strictListInference,
defaultSettings.strictListInference
),
// Use strict inference rules for dictionary expressions? // Use strict inference rules for dictionary expressions?
strictDictionaryInference: this._convertBoolean( strictDictionaryInference: this._convertBoolean(
configObj.strictDictionaryInference, DiagnosticRule.strictDictionaryInference, configObj.strictDictionaryInference,
defaultSettings.strictDictionaryInference), DiagnosticRule.strictDictionaryInference,
defaultSettings.strictDictionaryInference
),
// Should a None default value imply that the parameter type // Should a None default value imply that the parameter type
// is Optional? // is Optional?
strictParameterNoneValue: this._convertBoolean( strictParameterNoneValue: this._convertBoolean(
configObj.strictParameterNoneValue, DiagnosticRule.strictParameterNoneValue, configObj.strictParameterNoneValue,
defaultSettings.strictParameterNoneValue), DiagnosticRule.strictParameterNoneValue,
defaultSettings.strictParameterNoneValue
),
// Should "# type: ignore" be honored? // Should "# type: ignore" be honored?
enableTypeIgnoreComments: this._convertBoolean( enableTypeIgnoreComments: this._convertBoolean(
configObj.enableTypeIgnoreComments, DiagnosticRule.enableTypeIgnoreComments, configObj.enableTypeIgnoreComments,
defaultSettings.enableTypeIgnoreComments), DiagnosticRule.enableTypeIgnoreComments,
defaultSettings.enableTypeIgnoreComments
),
// Read the "reportTypeshedErrors" entry. // Read the "reportTypeshedErrors" entry.
reportTypeshedErrors: this._convertDiagnosticLevel( reportTypeshedErrors: this._convertDiagnosticLevel(
configObj.reportTypeshedErrors, DiagnosticRule.reportTypeshedErrors, configObj.reportTypeshedErrors,
defaultSettings.reportTypeshedErrors), DiagnosticRule.reportTypeshedErrors,
defaultSettings.reportTypeshedErrors
),
// Read the "reportMissingImports" entry. // Read the "reportMissingImports" entry.
reportMissingImports: this._convertDiagnosticLevel( reportMissingImports: this._convertDiagnosticLevel(
configObj.reportMissingImports, DiagnosticRule.reportMissingImports, configObj.reportMissingImports,
defaultSettings.reportMissingImports), DiagnosticRule.reportMissingImports,
defaultSettings.reportMissingImports
),
// Read the "reportUnusedImport" entry. // Read the "reportUnusedImport" entry.
reportUnusedImport: this._convertDiagnosticLevel( reportUnusedImport: this._convertDiagnosticLevel(
configObj.reportUnusedImport, DiagnosticRule.reportUnusedImport, configObj.reportUnusedImport,
defaultSettings.reportUnusedImport), DiagnosticRule.reportUnusedImport,
defaultSettings.reportUnusedImport
),
// Read the "reportUnusedClass" entry. // Read the "reportUnusedClass" entry.
reportUnusedClass: this._convertDiagnosticLevel( reportUnusedClass: this._convertDiagnosticLevel(
configObj.reportUnusedClass, DiagnosticRule.reportUnusedClass, configObj.reportUnusedClass,
defaultSettings.reportUnusedClass), DiagnosticRule.reportUnusedClass,
defaultSettings.reportUnusedClass
),
// Read the "reportUnusedFunction" entry. // Read the "reportUnusedFunction" entry.
reportUnusedFunction: this._convertDiagnosticLevel( reportUnusedFunction: this._convertDiagnosticLevel(
configObj.reportUnusedFunction, DiagnosticRule.reportUnusedFunction, configObj.reportUnusedFunction,
defaultSettings.reportUnusedFunction), DiagnosticRule.reportUnusedFunction,
defaultSettings.reportUnusedFunction
),
// Read the "reportUnusedVariable" entry. // Read the "reportUnusedVariable" entry.
reportUnusedVariable: this._convertDiagnosticLevel( reportUnusedVariable: this._convertDiagnosticLevel(
configObj.reportUnusedVariable, DiagnosticRule.reportUnusedVariable, configObj.reportUnusedVariable,
defaultSettings.reportUnusedVariable), DiagnosticRule.reportUnusedVariable,
defaultSettings.reportUnusedVariable
),
// Read the "reportDuplicateImport" entry. // Read the "reportDuplicateImport" entry.
reportDuplicateImport: this._convertDiagnosticLevel( reportDuplicateImport: this._convertDiagnosticLevel(
configObj.reportDuplicateImport, DiagnosticRule.reportDuplicateImport, configObj.reportDuplicateImport,
defaultSettings.reportDuplicateImport), DiagnosticRule.reportDuplicateImport,
defaultSettings.reportDuplicateImport
),
// Read the "reportMissingTypeStubs" entry. // Read the "reportMissingTypeStubs" entry.
reportMissingTypeStubs: this._convertDiagnosticLevel( reportMissingTypeStubs: this._convertDiagnosticLevel(
configObj.reportMissingTypeStubs, DiagnosticRule.reportMissingTypeStubs, configObj.reportMissingTypeStubs,
defaultSettings.reportMissingTypeStubs), DiagnosticRule.reportMissingTypeStubs,
defaultSettings.reportMissingTypeStubs
),
// Read the "reportImportCycles" entry. // Read the "reportImportCycles" entry.
reportImportCycles: this._convertDiagnosticLevel( reportImportCycles: this._convertDiagnosticLevel(
configObj.reportImportCycles, DiagnosticRule.reportImportCycles, configObj.reportImportCycles,
defaultSettings.reportImportCycles), DiagnosticRule.reportImportCycles,
defaultSettings.reportImportCycles
),
// Read the "reportOptionalSubscript" entry. // Read the "reportOptionalSubscript" entry.
reportOptionalSubscript: this._convertDiagnosticLevel( reportOptionalSubscript: this._convertDiagnosticLevel(
configObj.reportOptionalSubscript, DiagnosticRule.reportOptionalSubscript, configObj.reportOptionalSubscript,
defaultSettings.reportOptionalSubscript), DiagnosticRule.reportOptionalSubscript,
defaultSettings.reportOptionalSubscript
),
// Read the "reportOptionalMemberAccess" entry. // Read the "reportOptionalMemberAccess" entry.
reportOptionalMemberAccess: this._convertDiagnosticLevel( reportOptionalMemberAccess: this._convertDiagnosticLevel(
configObj.reportOptionalMemberAccess, DiagnosticRule.reportOptionalMemberAccess, configObj.reportOptionalMemberAccess,
defaultSettings.reportOptionalMemberAccess), DiagnosticRule.reportOptionalMemberAccess,
defaultSettings.reportOptionalMemberAccess
),
// Read the "reportOptionalCall" entry. // Read the "reportOptionalCall" entry.
reportOptionalCall: this._convertDiagnosticLevel( reportOptionalCall: this._convertDiagnosticLevel(
configObj.reportOptionalCall, DiagnosticRule.reportOptionalCall, configObj.reportOptionalCall,
defaultSettings.reportOptionalCall), DiagnosticRule.reportOptionalCall,
defaultSettings.reportOptionalCall
),
// Read the "reportOptionalIterable" entry. // Read the "reportOptionalIterable" entry.
reportOptionalIterable: this._convertDiagnosticLevel( reportOptionalIterable: this._convertDiagnosticLevel(
configObj.reportOptionalIterable, DiagnosticRule.reportOptionalIterable, configObj.reportOptionalIterable,
defaultSettings.reportOptionalIterable), DiagnosticRule.reportOptionalIterable,
defaultSettings.reportOptionalIterable
),
// Read the "reportOptionalContextManager" entry. // Read the "reportOptionalContextManager" entry.
reportOptionalContextManager: this._convertDiagnosticLevel( reportOptionalContextManager: this._convertDiagnosticLevel(
configObj.reportOptionalContextManager, DiagnosticRule.reportOptionalContextManager, configObj.reportOptionalContextManager,
defaultSettings.reportOptionalContextManager), DiagnosticRule.reportOptionalContextManager,
defaultSettings.reportOptionalContextManager
),
// Read the "reportOptionalOperand" entry. // Read the "reportOptionalOperand" entry.
reportOptionalOperand: this._convertDiagnosticLevel( reportOptionalOperand: this._convertDiagnosticLevel(
configObj.reportOptionalOperand, DiagnosticRule.reportOptionalOperand, configObj.reportOptionalOperand,
defaultSettings.reportOptionalOperand), DiagnosticRule.reportOptionalOperand,
defaultSettings.reportOptionalOperand
),
// Read the "reportUntypedFunctionDecorator" entry. // Read the "reportUntypedFunctionDecorator" entry.
reportUntypedFunctionDecorator: this._convertDiagnosticLevel( reportUntypedFunctionDecorator: this._convertDiagnosticLevel(
configObj.reportUntypedFunctionDecorator, DiagnosticRule.reportUntypedFunctionDecorator, configObj.reportUntypedFunctionDecorator,
defaultSettings.reportUntypedFunctionDecorator), DiagnosticRule.reportUntypedFunctionDecorator,
defaultSettings.reportUntypedFunctionDecorator
),
// Read the "reportUntypedClassDecorator" entry. // Read the "reportUntypedClassDecorator" entry.
reportUntypedClassDecorator: this._convertDiagnosticLevel( reportUntypedClassDecorator: this._convertDiagnosticLevel(
configObj.reportUntypedClassDecorator, DiagnosticRule.reportUntypedClassDecorator, configObj.reportUntypedClassDecorator,
defaultSettings.reportUntypedClassDecorator), DiagnosticRule.reportUntypedClassDecorator,
defaultSettings.reportUntypedClassDecorator
),
// Read the "reportUntypedBaseClass" entry. // Read the "reportUntypedBaseClass" entry.
reportUntypedBaseClass: this._convertDiagnosticLevel( reportUntypedBaseClass: this._convertDiagnosticLevel(
configObj.reportUntypedBaseClass, DiagnosticRule.reportUntypedBaseClass, configObj.reportUntypedBaseClass,
defaultSettings.reportUntypedBaseClass), DiagnosticRule.reportUntypedBaseClass,
defaultSettings.reportUntypedBaseClass
),
// Read the "reportUntypedNamedTuple" entry. // Read the "reportUntypedNamedTuple" entry.
reportUntypedNamedTuple: this._convertDiagnosticLevel( reportUntypedNamedTuple: this._convertDiagnosticLevel(
configObj.reportUntypedNamedTuple, DiagnosticRule.reportUntypedNamedTuple, configObj.reportUntypedNamedTuple,
defaultSettings.reportUntypedNamedTuple), DiagnosticRule.reportUntypedNamedTuple,
defaultSettings.reportUntypedNamedTuple
),
// Read the "reportPrivateUsage" entry. // Read the "reportPrivateUsage" entry.
reportPrivateUsage: this._convertDiagnosticLevel( reportPrivateUsage: this._convertDiagnosticLevel(
configObj.reportPrivateUsage, DiagnosticRule.reportPrivateUsage, configObj.reportPrivateUsage,
defaultSettings.reportPrivateUsage), DiagnosticRule.reportPrivateUsage,
defaultSettings.reportPrivateUsage
),
// Read the "reportConstantRedefinition" entry. // Read the "reportConstantRedefinition" entry.
reportConstantRedefinition: this._convertDiagnosticLevel( reportConstantRedefinition: this._convertDiagnosticLevel(
configObj.reportConstantRedefinition, DiagnosticRule.reportConstantRedefinition, configObj.reportConstantRedefinition,
defaultSettings.reportConstantRedefinition), DiagnosticRule.reportConstantRedefinition,
defaultSettings.reportConstantRedefinition
),
// Read the "reportIncompatibleMethodOverride" entry. // Read the "reportIncompatibleMethodOverride" entry.
reportIncompatibleMethodOverride: this._convertDiagnosticLevel( reportIncompatibleMethodOverride: this._convertDiagnosticLevel(
configObj.reportIncompatibleMethodOverride, DiagnosticRule.reportIncompatibleMethodOverride, configObj.reportIncompatibleMethodOverride,
defaultSettings.reportIncompatibleMethodOverride), DiagnosticRule.reportIncompatibleMethodOverride,
defaultSettings.reportIncompatibleMethodOverride
),
// Read the "reportInvalidStringEscapeSequence" entry. // Read the "reportInvalidStringEscapeSequence" entry.
reportInvalidStringEscapeSequence: this._convertDiagnosticLevel( reportInvalidStringEscapeSequence: this._convertDiagnosticLevel(
configObj.reportInvalidStringEscapeSequence, DiagnosticRule.reportInvalidStringEscapeSequence, configObj.reportInvalidStringEscapeSequence,
defaultSettings.reportInvalidStringEscapeSequence), DiagnosticRule.reportInvalidStringEscapeSequence,
defaultSettings.reportInvalidStringEscapeSequence
),
// Read the "reportUnknownParameterType" entry. // Read the "reportUnknownParameterType" entry.
reportUnknownParameterType: this._convertDiagnosticLevel( reportUnknownParameterType: this._convertDiagnosticLevel(
configObj.reportUnknownParameterType, DiagnosticRule.reportUnknownParameterType, configObj.reportUnknownParameterType,
defaultSettings.reportUnknownParameterType), DiagnosticRule.reportUnknownParameterType,
defaultSettings.reportUnknownParameterType
),
// Read the "reportUnknownArgumentType" entry. // Read the "reportUnknownArgumentType" entry.
reportUnknownArgumentType: this._convertDiagnosticLevel( reportUnknownArgumentType: this._convertDiagnosticLevel(
configObj.reportUnknownArgumentType, DiagnosticRule.reportUnknownArgumentType, configObj.reportUnknownArgumentType,
defaultSettings.reportUnknownArgumentType), DiagnosticRule.reportUnknownArgumentType,
defaultSettings.reportUnknownArgumentType
),
// Read the "reportUnknownLambdaType" entry. // Read the "reportUnknownLambdaType" entry.
reportUnknownLambdaType: this._convertDiagnosticLevel( reportUnknownLambdaType: this._convertDiagnosticLevel(
configObj.reportUnknownLambdaType, DiagnosticRule.reportUnknownLambdaType, configObj.reportUnknownLambdaType,
defaultSettings.reportUnknownLambdaType), DiagnosticRule.reportUnknownLambdaType,
defaultSettings.reportUnknownLambdaType
),
// Read the "reportUnknownVariableType" entry. // Read the "reportUnknownVariableType" entry.
reportUnknownVariableType: this._convertDiagnosticLevel( reportUnknownVariableType: this._convertDiagnosticLevel(
configObj.reportUnknownVariableType, DiagnosticRule.reportUnknownVariableType, configObj.reportUnknownVariableType,
defaultSettings.reportUnknownVariableType), DiagnosticRule.reportUnknownVariableType,
defaultSettings.reportUnknownVariableType
),
// Read the "reportUnknownMemberType" entry. // Read the "reportUnknownMemberType" entry.
reportUnknownMemberType: this._convertDiagnosticLevel( reportUnknownMemberType: this._convertDiagnosticLevel(
configObj.reportUnknownMemberType, DiagnosticRule.reportUnknownMemberType, configObj.reportUnknownMemberType,
defaultSettings.reportUnknownMemberType), DiagnosticRule.reportUnknownMemberType,
defaultSettings.reportUnknownMemberType
),
// Read the "reportCallInDefaultInitializer" entry. // Read the "reportCallInDefaultInitializer" entry.
reportCallInDefaultInitializer: this._convertDiagnosticLevel( reportCallInDefaultInitializer: this._convertDiagnosticLevel(
configObj.reportCallInDefaultInitializer, DiagnosticRule.reportCallInDefaultInitializer, configObj.reportCallInDefaultInitializer,
defaultSettings.reportCallInDefaultInitializer), DiagnosticRule.reportCallInDefaultInitializer,
defaultSettings.reportCallInDefaultInitializer
),
// Read the "reportUnnecessaryIsInstance" entry. // Read the "reportUnnecessaryIsInstance" entry.
reportUnnecessaryIsInstance: this._convertDiagnosticLevel( reportUnnecessaryIsInstance: this._convertDiagnosticLevel(
configObj.reportUnnecessaryIsInstance, DiagnosticRule.reportUnnecessaryIsInstance, configObj.reportUnnecessaryIsInstance,
defaultSettings.reportUnnecessaryIsInstance), DiagnosticRule.reportUnnecessaryIsInstance,
defaultSettings.reportUnnecessaryIsInstance
),
// Read the "reportUnnecessaryCast" entry. // Read the "reportUnnecessaryCast" entry.
reportUnnecessaryCast: this._convertDiagnosticLevel( reportUnnecessaryCast: this._convertDiagnosticLevel(
configObj.reportUnnecessaryCast, DiagnosticRule.reportUnnecessaryCast, configObj.reportUnnecessaryCast,
defaultSettings.reportUnnecessaryCast), DiagnosticRule.reportUnnecessaryCast,
defaultSettings.reportUnnecessaryCast
),
// Read the "reportAssertAlwaysTrue" entry. // Read the "reportAssertAlwaysTrue" entry.
reportAssertAlwaysTrue: this._convertDiagnosticLevel( reportAssertAlwaysTrue: this._convertDiagnosticLevel(
configObj.reportAssertAlwaysTrue, DiagnosticRule.reportAssertAlwaysTrue, configObj.reportAssertAlwaysTrue,
defaultSettings.reportAssertAlwaysTrue), DiagnosticRule.reportAssertAlwaysTrue,
defaultSettings.reportAssertAlwaysTrue
),
// Read the "reportSelfClsParameterName" entry. // Read the "reportSelfClsParameterName" entry.
reportSelfClsParameterName: this._convertDiagnosticLevel( reportSelfClsParameterName: this._convertDiagnosticLevel(
configObj.reportSelfClsParameterName, DiagnosticRule.reportSelfClsParameterName, configObj.reportSelfClsParameterName,
defaultSettings.reportSelfClsParameterName), DiagnosticRule.reportSelfClsParameterName,
defaultSettings.reportSelfClsParameterName
),
// Read the "reportImplicitStringConcatenation" entry. // Read the "reportImplicitStringConcatenation" entry.
reportImplicitStringConcatenation: this._convertDiagnosticLevel( reportImplicitStringConcatenation: this._convertDiagnosticLevel(
configObj.reportImplicitStringConcatenation, DiagnosticRule.reportImplicitStringConcatenation, configObj.reportImplicitStringConcatenation,
defaultSettings.reportImplicitStringConcatenation) DiagnosticRule.reportImplicitStringConcatenation,
defaultSettings.reportImplicitStringConcatenation
)
}; };
// Read the "venvPath". // Read the "venvPath".
@ -736,9 +807,9 @@ export class ConfigOptions {
if (typeof configObj.typeshedPath !== 'string') { if (typeof configObj.typeshedPath !== 'string') {
console.log(`Config "typeshedPath" field must contain a string.`); console.log(`Config "typeshedPath" field must contain a string.`);
} else { } else {
this.typeshedPath = configObj.typeshedPath ? this.typeshedPath = configObj.typeshedPath
normalizePath(combinePaths(this.projectRoot, configObj.typeshedPath)) : ? normalizePath(combinePaths(this.projectRoot, configObj.typeshedPath))
''; : '';
} }
} }
@ -781,9 +852,7 @@ export class ConfigOptions {
return defaultValue; return defaultValue;
} }
private _convertDiagnosticLevel(value: any, fieldName: string, private _convertDiagnosticLevel(value: any, fieldName: string, defaultValue: DiagnosticLevel): DiagnosticLevel {
defaultValue: DiagnosticLevel): DiagnosticLevel {
if (value === undefined) { if (value === undefined) {
return defaultValue; return defaultValue;
} else if (typeof value === 'boolean') { } else if (typeof value === 'boolean') {
@ -798,11 +867,17 @@ export class ConfigOptions {
return defaultValue; return defaultValue;
} }
private _initExecutionEnvironmentFromJson(envObj: any, index: number, private _initExecutionEnvironmentFromJson(
console: ConsoleInterface): ExecutionEnvironment | undefined { envObj: any,
index: number,
console: ConsoleInterface
): ExecutionEnvironment | undefined {
try { try {
const newExecEnv = new ExecutionEnvironment(this.projectRoot, const newExecEnv = new ExecutionEnvironment(
this.defaultPythonVersion, this.defaultPythonPlatform); this.projectRoot,
this.defaultPythonVersion,
this.defaultPythonPlatform
);
// Validate the root. // Validate the root.
if (envObj.root && typeof envObj.root === 'string') { if (envObj.root && typeof envObj.root === 'string') {
@ -819,8 +894,10 @@ export class ConfigOptions {
const pathList = envObj.extraPaths as string[]; const pathList = envObj.extraPaths as string[];
pathList.forEach((path, pathIndex) => { pathList.forEach((path, pathIndex) => {
if (typeof path !== 'string') { if (typeof path !== 'string') {
console.log(`Config executionEnvironments index ${ index }:` + console.log(
` extraPaths field ${ pathIndex } must be a string.`); `Config executionEnvironments index ${index}:` +
` extraPaths field ${pathIndex} must be a string.`
);
} else { } else {
newExecEnv.extraPaths.push(normalizePath(combinePaths(this.projectRoot, path))); newExecEnv.extraPaths.push(normalizePath(combinePaths(this.projectRoot, path)));
} }

38
server/src/common/core.ts
View File

@ -21,32 +21,48 @@ export const enum Comparison {
export type AnyFunction = (...args: never[]) => void; export type AnyFunction = (...args: never[]) => void;
/** Do nothing and return false */ /** Do nothing and return false */
export function returnFalse(): false { return false; } export function returnFalse(): false {
return false;
}
/** Do nothing and return true */ /** Do nothing and return true */
export function returnTrue(): true { return true; } export function returnTrue(): true {
return true;
}
/** Do nothing and return undefined */ /** Do nothing and return undefined */
export function returnUndefined(): undefined { return undefined; } export function returnUndefined(): undefined {
return undefined;
}
/** Returns its argument. */ /** Returns its argument. */
export function identity<T>(x: T) { return x; } export function identity<T>(x: T) {
return x;
}
/** Returns lower case string */ /** Returns lower case string */
export function toLowerCase(x: string) { return x.toLowerCase(); } export function toLowerCase(x: string) {
return x.toLowerCase();
}
export function equateValues<T>(a: T, b: T) { return a === b; } export function equateValues<T>(a: T, b: T) {
return a === b;
}
export type GetCanonicalFileName = (fileName: string) => string; export type GetCanonicalFileName = (fileName: string) => string;
export function compareComparableValues(a: string | undefined, b: string | undefined): Comparison; export function compareComparableValues(a: string | undefined, b: string | undefined): Comparison;
export function compareComparableValues(a: number | undefined, b: number | undefined): Comparison; export function compareComparableValues(a: number | undefined, b: number | undefined): Comparison;
export function compareComparableValues(a: string | number | undefined, b: string | number | undefined) { export function compareComparableValues(a: string | number | undefined, b: string | number | undefined) {
return a === b ? Comparison.EqualTo : return a === b
a === undefined ? Comparison.LessThan : ? Comparison.EqualTo
b === undefined ? Comparison.GreaterThan : : a === undefined
a < b ? Comparison.LessThan : ? Comparison.LessThan
Comparison.GreaterThan; : b === undefined
? Comparison.GreaterThan
: a < b
? Comparison.LessThan
: Comparison.GreaterThan;
} }
/** /**

17
server/src/common/debug.ts
View File

@ -9,12 +9,17 @@
import { stableSort } from './collectionUtils'; import { stableSort } from './collectionUtils';
import { AnyFunction, compareValues, hasProperty } from './core'; import { AnyFunction, compareValues, hasProperty } from './core';
export function assert(expression: boolean, message?: string, export function assert(
verboseDebugInfo?: string | (() => string), stackCrawlMark?: AnyFunction): void { expression: boolean,
message?: string,
verboseDebugInfo?: string | (() => string),
stackCrawlMark?: AnyFunction
): void {
if (!expression) { if (!expression) {
if (verboseDebugInfo) { if (verboseDebugInfo) {
message += '\r\nVerbose Debug Information: ' + (typeof verboseDebugInfo === 'string' ? verboseDebugInfo : verboseDebugInfo()); message +=
'\r\nVerbose Debug Information: ' +
(typeof verboseDebugInfo === 'string' ? verboseDebugInfo : verboseDebugInfo());
} }
fail(message ? 'False expression: ' + message : 'False expression.', stackCrawlMark || assert); fail(message ? 'False expression: ' + message : 'False expression.', stackCrawlMark || assert);
} }
@ -30,7 +35,9 @@ export function fail(message?: string, stackCrawlMark?: AnyFunction): never {
} }
export function assertDefined<T>(value: T | null | undefined, message?: string): T { export function assertDefined<T>(value: T | null | undefined, message?: string): T {
if (value === undefined || value === null) { return fail(message); } if (value === undefined || value === null) {
return fail(message);
}
return value; return value;
} }

4
server/src/common/diagnostic.ts
View File

@ -42,9 +42,7 @@ export class Diagnostic {
private _rule: string | undefined; private _rule: string | undefined;
private _relatedInfo: DiagnosticRelatedInfo[] = []; private _relatedInfo: DiagnosticRelatedInfo[] = [];
constructor(readonly category: DiagnosticCategory, readonly message: string, constructor(readonly category: DiagnosticCategory, readonly message: string, readonly range: Range) {}
readonly range: Range) {
}
addAction(action: DiagnosticAction) { addAction(action: DiagnosticAction) {
if (this._actions === undefined) { if (this._actions === undefined) {

2
server/src/common/editAction.ts
View File

@ -7,7 +7,7 @@
* Represents a single edit within a file. * Represents a single edit within a file.
*/ */
import { Range } from "./textRange"; import { Range } from './textRange';
export interface TextEditAction { export interface TextEditAction {
range: Range; range: Range;

192
server/src/common/pathUtils.ts
View File

@ -15,8 +15,11 @@ import { some } from './collectionUtils';
import { compareValues, Comparison, GetCanonicalFileName, identity } from './core'; import { compareValues, Comparison, GetCanonicalFileName, identity } from './core';
import * as debug from './debug'; import * as debug from './debug';
import { import {
compareStringsCaseInsensitive, compareStringsCaseSensitive, equateStringsCaseInsensitive, compareStringsCaseInsensitive,
equateStringsCaseSensitive, getStringComparer compareStringsCaseSensitive,
equateStringsCaseInsensitive,
equateStringsCaseSensitive,
getStringComparer
} from './stringUtils'; } from './stringUtils';
import { VirtualFileSystem } from './vfs'; import { VirtualFileSystem } from './vfs';
@ -36,7 +39,10 @@ export interface FileSystemEntries {
directories: string[]; directories: string[];
} }
export function forEachAncestorDirectory(directory: string, callback: (directory: string) => string | undefined): string | undefined { export function forEachAncestorDirectory(
directory: string,
callback: (directory: string) => string | undefined
): string | undefined {
while (true) { while (true) {
const result = callback(directory); const result = callback(directory);
if (result !== undefined) { if (result !== undefined) {
@ -58,15 +64,23 @@ export function getDirectoryPath(pathString: string): string {
export function getRootLength(pathString: string): number { export function getRootLength(pathString: string): number {
if (pathString.charAt(0) === path.sep) { if (pathString.charAt(0) === path.sep) {
if (pathString.charAt(1) !== path.sep) { return 1; } if (pathString.charAt(1) !== path.sep) {
return 1;
}
const p1 = pathString.indexOf(path.sep, 2); const p1 = pathString.indexOf(path.sep, 2);
if (p1 < 0) { return 2; } if (p1 < 0) {
return 2;
}
const p2 = pathString.indexOf(path.sep, p1 + 1); const p2 = pathString.indexOf(path.sep, p1 + 1);
if (p2 < 0) { return p1 + 1; } if (p2 < 0) {
return p1 + 1;
}
return p2 + 1; return p2 + 1;
} }
if (pathString.charAt(1) === ':') { if (pathString.charAt(1) === ':') {
if (pathString.charAt(2) === path.sep) { return 3; } if (pathString.charAt(2) === path.sep) {
return 3;
}
} }
return 0; return 0;
} }
@ -84,7 +98,9 @@ export function getPathComponents(pathString: string) {
} }
export function reducePathComponents(components: readonly string[]) { export function reducePathComponents(components: readonly string[]) {
if (!some(components)) { return []; } if (!some(components)) {
return [];
}
// Reduce the path components by eliminating // Reduce the path components by eliminating
// any '.' or '..'. // any '.' or '..'.
@ -112,7 +128,9 @@ export function reducePathComponents(components: readonly string[]) {
} }
export function combinePathComponents(components: string[]): string { export function combinePathComponents(components: string[]): string {
if (components.length === 0) { return ''; } if (components.length === 0) {
return '';
}
const root = components[0] && ensureTrailingDirectorySeparator(components[0]); const root = components[0] && ensureTrailingDirectorySeparator(components[0]);
return normalizeSlashes(root + components.slice(1).join(path.sep)); return normalizeSlashes(root + components.slice(1).join(path.sep));
@ -158,7 +176,9 @@ export function getFileSize(fs: VirtualFileSystem, path: string) {
if (stat.isFile()) { if (stat.isFile()) {
return stat.size; return stat.size;
} }
} catch { /*ignore*/ } } catch {
/*ignore*/
}
return 0; return 0;
} }
@ -242,8 +262,12 @@ export function containsPath(parent: string, child: string, currentDirectory?: s
ignoreCase = currentDirectory; ignoreCase = currentDirectory;
} }
if (parent === undefined || child === undefined) { return false; } if (parent === undefined || child === undefined) {
if (parent === child) { return true; } return false;
}
if (parent === child) {
return true;
}
const parentComponents = getPathComponents(parent); const parentComponents = getPathComponents(parent);
const childComponents = getPathComponents(child); const childComponents = getPathComponents(child);
@ -281,10 +305,22 @@ export function changeAnyExtension(path: string, ext: string): string;
* changeAnyExtension("/path/to/file.ext", ".js", [".ext", ".ts"]) === "/path/to/file.js" * changeAnyExtension("/path/to/file.ext", ".js", [".ext", ".ts"]) === "/path/to/file.js"
* ``` * ```
*/ */
export function changeAnyExtension(path: string, ext: string, extensions: string | readonly string[], ignoreCase: boolean): string; export function changeAnyExtension(
export function changeAnyExtension(path: string, ext: string, extensions?: string | readonly string[], ignoreCase?: boolean): string { path: string,
const pathext = extensions !== undefined && ignoreCase !== undefined ? ext: string,
getAnyExtensionFromPath(path, extensions, ignoreCase) : getAnyExtensionFromPath(path); extensions: string | readonly string[],
ignoreCase: boolean
): string;
export function changeAnyExtension(
path: string,
ext: string,
extensions?: string | readonly string[],
ignoreCase?: boolean
): string {
const pathext =
extensions !== undefined && ignoreCase !== undefined
? getAnyExtensionFromPath(path, extensions, ignoreCase)
: getAnyExtensionFromPath(path);
return pathext ? path.slice(0, path.length - pathext.length) + (ext.startsWith('.') ? ext : '.' + ext) : path; return pathext ? path.slice(0, path.length - pathext.length) + (ext.startsWith('.') ? ext : '.' + ext) : path;
} }
@ -309,13 +345,24 @@ export function getAnyExtensionFromPath(path: string): string;
* getAnyExtensionFromPath("/path/to/file.js", [".ext", ".js"], true) === ".js" * getAnyExtensionFromPath("/path/to/file.js", [".ext", ".js"], true) === ".js"
* getAnyExtensionFromPath("/path/to/file.ext", ".EXT", false) === "" * getAnyExtensionFromPath("/path/to/file.ext", ".EXT", false) === ""
*/ */
export function getAnyExtensionFromPath(path: string, extensions: string | readonly string[], ignoreCase: boolean): string; export function getAnyExtensionFromPath(
export function getAnyExtensionFromPath(path: string, extensions?: string | readonly string[], ignoreCase?: boolean): string { path: string,
extensions: string | readonly string[],
ignoreCase: boolean
): string;
export function getAnyExtensionFromPath(
path: string,
extensions?: string | readonly string[],
ignoreCase?: boolean
): string {
// Retrieves any string from the final "." onwards from a base file name. // Retrieves any string from the final "." onwards from a base file name.
// Unlike extensionFromPath, which throws an exception on unrecognized extensions. // Unlike extensionFromPath, which throws an exception on unrecognized extensions.
if (extensions) { if (extensions) {
return getAnyExtensionFromPathWorker(stripTrailingDirectorySeparator(path), extensions, return getAnyExtensionFromPathWorker(
ignoreCase ? equateStringsCaseInsensitive : equateStringsCaseSensitive); stripTrailingDirectorySeparator(path),
extensions,
ignoreCase ? equateStringsCaseInsensitive : equateStringsCaseSensitive
);
} }
const baseFileName = getBaseFileName(path); const baseFileName = getBaseFileName(path);
const extensionIndex = baseFileName.lastIndexOf('.'); const extensionIndex = baseFileName.lastIndexOf('.');
@ -354,20 +401,28 @@ export function getBaseFileName(pathString: string): string;
* getBaseFileName("/path/to/file.ext", ".EXT", false) === "file.ext" * getBaseFileName("/path/to/file.ext", ".EXT", false) === "file.ext"
* ``` * ```
*/ */
export function getBaseFileName(pathString: string, extensions: string | readonly string[], ignoreCase: boolean): string; export function getBaseFileName(
pathString: string,
extensions: string | readonly string[],
ignoreCase: boolean
): string;
export function getBaseFileName(pathString: string, extensions?: string | readonly string[], ignoreCase?: boolean) { export function getBaseFileName(pathString: string, extensions?: string | readonly string[], ignoreCase?: boolean) {
pathString = normalizeSlashes(pathString); pathString = normalizeSlashes(pathString);
// if the path provided is itself the root, then it has not file name. // if the path provided is itself the root, then it has not file name.
const rootLength = getRootLength(pathString); const rootLength = getRootLength(pathString);
if (rootLength === pathString.length) { return ''; } if (rootLength === pathString.length) {
return '';
}
// return the trailing portion of the path starting after the last (non-terminal) directory // return the trailing portion of the path starting after the last (non-terminal) directory
// separator but not including any trailing directory separator. // separator but not including any trailing directory separator.
pathString = stripTrailingDirectorySeparator(pathString); pathString = stripTrailingDirectorySeparator(pathString);
const name = pathString.slice(Math.max(getRootLength(pathString), pathString.lastIndexOf(path.sep) + 1)); const name = pathString.slice(Math.max(getRootLength(pathString), pathString.lastIndexOf(path.sep) + 1));
const extension = extensions !== undefined && ignoreCase !== undefined ? const extension =
getAnyExtensionFromPath(name, extensions, ignoreCase) : undefined; extensions !== undefined && ignoreCase !== undefined
? getAnyExtensionFromPath(name, extensions, ignoreCase)
: undefined;
return extension ? name.slice(0, name.length - extension.length) : name; return extension ? name.slice(0, name.length - extension.length) : name;
} }
@ -379,15 +434,29 @@ export function getRelativePathFromDirectory(from: string, to: string, ignoreCas
/** /**
* Gets a relative path that can be used to traverse between `from` and `to`. * Gets a relative path that can be used to traverse between `from` and `to`.
*/ */
export function getRelativePathFromDirectory(fromDirectory: string, to: string, getCanonicalFileName: GetCanonicalFileName): string; export function getRelativePathFromDirectory(
export function getRelativePathFromDirectory(fromDirectory: string, to: string, fromDirectory: string,
getCanonicalFileNameOrIgnoreCase: GetCanonicalFileName | boolean) { to: string,
getCanonicalFileName: GetCanonicalFileName
debug.assert((getRootLength(fromDirectory) > 0) === (getRootLength(to) > 0), 'Paths must either both be absolute or both be relative'); ): string;
const getCanonicalFileName = typeof getCanonicalFileNameOrIgnoreCase === 'function' ? getCanonicalFileNameOrIgnoreCase : identity; export function getRelativePathFromDirectory(
fromDirectory: string,
to: string,
getCanonicalFileNameOrIgnoreCase: GetCanonicalFileName | boolean
) {
debug.assert(
getRootLength(fromDirectory) > 0 === getRootLength(to) > 0,
'Paths must either both be absolute or both be relative'
);
const getCanonicalFileName =
typeof getCanonicalFileNameOrIgnoreCase === 'function' ? getCanonicalFileNameOrIgnoreCase : identity;
const ignoreCase = typeof getCanonicalFileNameOrIgnoreCase === 'boolean' ? getCanonicalFileNameOrIgnoreCase : false; const ignoreCase = typeof getCanonicalFileNameOrIgnoreCase === 'boolean' ? getCanonicalFileNameOrIgnoreCase : false;
const pathComponents = getPathComponentsRelativeTo(fromDirectory, to, ignoreCase ? const pathComponents = getPathComponentsRelativeTo(
equateStringsCaseInsensitive : equateStringsCaseSensitive, getCanonicalFileName); fromDirectory,
to,
ignoreCase ? equateStringsCaseInsensitive : equateStringsCaseSensitive,
getCanonicalFileName
);
return combinePathComponents(pathComponents); return combinePathComponents(pathComponents);
} }
@ -532,8 +601,7 @@ export function getWildcardRegexPattern(rootPath: string, fileSpec: string): str
component = escapedSeparator + component; component = escapedSeparator + component;
} }
regExPattern += component.replace( regExPattern += component.replace(reservedCharacterPattern, match => {
reservedCharacterPattern, match => {
if (match === '*') { if (match === '*') {
return `[^${escapedSeparator}]*`; return `[^${escapedSeparator}]*`;
} else if (match === '?') { } else if (match === '?') {
@ -625,9 +693,15 @@ export function isDiskPathRoot(path: string) {
//// Path Comparisons //// Path Comparisons
function comparePathsWorker(a: string, b: string, componentComparer: (a: string, b: string) => Comparison) { function comparePathsWorker(a: string, b: string, componentComparer: (a: string, b: string) => Comparison) {
if (a === b) { return Comparison.EqualTo; } if (a === b) {
if (a === undefined) { return Comparison.LessThan; } return Comparison.EqualTo;
if (b === undefined) { return Comparison.GreaterThan; } }
if (a === undefined) {
return Comparison.LessThan;
}
if (b === undefined) {
return Comparison.GreaterThan;
}
// NOTE: Performance optimization - shortcut if the root segments differ as there would be no // NOTE: Performance optimization - shortcut if the root segments differ as there would be no
// need to perform path reduction. // need to perform path reduction.
@ -665,21 +739,31 @@ function comparePathsWorker(a: string, b: string, componentComparer: (a: string,
return compareValues(aComponents.length, bComponents.length); return compareValues(aComponents.length, bComponents.length);
} }
function getAnyExtensionFromPathWorker(path: string, extensions: string | readonly string[], function getAnyExtensionFromPathWorker(
stringEqualityComparer: (a: string, b: string) => boolean) { path: string,
extensions: string | readonly string[],
stringEqualityComparer: (a: string, b: string) => boolean
) {
if (typeof extensions === 'string') { if (typeof extensions === 'string') {
return tryGetExtensionFromPath(path, extensions, stringEqualityComparer) || ''; return tryGetExtensionFromPath(path, extensions, stringEqualityComparer) || '';
} }
for (const extension of extensions) { for (const extension of extensions) {
const result = tryGetExtensionFromPath(path, extension, stringEqualityComparer); const result = tryGetExtensionFromPath(path, extension, stringEqualityComparer);
if (result) { return result; } if (result) {
return result;
}
} }
return ''; return '';
} }
function tryGetExtensionFromPath(path: string, extension: string, stringEqualityComparer: (a: string, b: string) => boolean) { function tryGetExtensionFromPath(
if (!extension.startsWith('.')) { extension = '.' + extension; } path: string,
extension: string,
stringEqualityComparer: (a: string, b: string) => boolean
) {
if (!extension.startsWith('.')) {
extension = '.' + extension;
}
if (path.length >= extension.length && path.charCodeAt(path.length - extension.length) === Char.Period) { if (path.length >= extension.length && path.charCodeAt(path.length - extension.length) === Char.Period) {
const pathExtension = path.slice(path.length - extension.length); const pathExtension = path.slice(path.length - extension.length);
if (stringEqualityComparer(pathExtension, extension)) { if (stringEqualityComparer(pathExtension, extension)) {
@ -690,9 +774,12 @@ function tryGetExtensionFromPath(path: string, extension: string, stringEquality
return undefined; return undefined;
} }
function getPathComponentsRelativeTo(from: string, to: string, stringEqualityComparer: (a: string, b: string) => boolean, function getPathComponentsRelativeTo(
getCanonicalFileName: GetCanonicalFileName) { from: string,
to: string,
stringEqualityComparer: (a: string, b: string) => boolean,
getCanonicalFileName: GetCanonicalFileName
) {
const fromComponents = getPathComponents(from); const fromComponents = getPathComponents(from);
const toComponents = getPathComponents(to); const toComponents = getPathComponents(to);
@ -701,7 +788,9 @@ function getPathComponentsRelativeTo(from: string, to: string, stringEqualityCom
const fromComponent = getCanonicalFileName(fromComponents[start]); const fromComponent = getCanonicalFileName(fromComponents[start]);
const toComponent = getCanonicalFileName(toComponents[start]); const toComponent = getCanonicalFileName(toComponents[start]);
const comparer = start === 0 ? equateStringsCaseInsensitive : stringEqualityComparer; const comparer = start === 0 ? equateStringsCaseInsensitive : stringEqualityComparer;
if (!comparer(fromComponent, toComponent)) { break; } if (!comparer(fromComponent, toComponent)) {
break;
}
} }
if (start === 0) { if (start === 0) {
@ -725,9 +814,12 @@ function fileSystemEntryExists(fs: VirtualFileSystem, path: string, entryKind: F
try { try {
const stat = fs.statSync(path); const stat = fs.statSync(path);
switch (entryKind) { switch (entryKind) {
case FileSystemEntryKind.File: return stat.isFile(); case FileSystemEntryKind.File:
case FileSystemEntryKind.Directory: return stat.isDirectory(); return stat.isFile();
default: return false; case FileSystemEntryKind.Directory:
return stat.isDirectory();
default:
return false;
} }
} catch (e) { } catch (e) {
return false; return false;

10
server/src/common/positionUtils.ts
View File

@ -38,16 +38,18 @@ export function convertOffsetToPosition(offset: number, lines: TextRangeCollecti
} }
// Translates a start/end file offset into a pair of line/column positions. // Translates a start/end file offset into a pair of line/column positions.
export function convertOffsetsToRange(startOffset: number, endOffset: number, export function convertOffsetsToRange(
lines: TextRangeCollection<TextRange>): Range { startOffset: number,
endOffset: number,
lines: TextRangeCollection<TextRange>
): Range {
const start = convertOffsetToPosition(startOffset, lines); const start = convertOffsetToPosition(startOffset, lines);
const end = convertOffsetToPosition(endOffset, lines); const end = convertOffsetToPosition(endOffset, lines);
return { start, end }; return { start, end };
} }
// Translates a position (line and col) into a file offset. // Translates a position (line and col) into a file offset.
export function convertPositionToOffset(position: Position, export function convertPositionToOffset(position: Position, lines: TextRangeCollection<TextRange>): number | undefined {
lines: TextRangeCollection<TextRange>): number | undefined {
if (position.line >= lines.count) { if (position.line >= lines.count) {
return undefined; return undefined;
} }

6
server/src/common/pythonVersion.ts
View File

@ -31,8 +31,8 @@ export const latestStablePythonVersion = PythonVersion.V38;
export const latestPythonVersion = PythonVersion.V38; export const latestPythonVersion = PythonVersion.V38;
export function versionToString(version: PythonVersion): string { export function versionToString(version: PythonVersion): string {
const majorVersion = (version >> 8) & 0xFF; const majorVersion = (version >> 8) & 0xff;
const minorVersion = version & 0xFF; const minorVersion = version & 0xff;
return `${majorVersion}.${minorVersion}`; return `${majorVersion}.${minorVersion}`;
} }
@ -67,5 +67,5 @@ export function versionFromString(verString: string): PythonVersion | undefined
} }
export function is3x(version: PythonVersion): boolean { export function is3x(version: PythonVersion): boolean {
return (version >> 8) === 3; return version >> 8 === 3;
} }

3
server/src/common/textRange.ts
View File

@ -108,8 +108,7 @@ export function doRangesOverlap(a: Range, b: Range) {
} }
export function doesRangeContain(range: Range, position: Position) { export function doesRangeContain(range: Range, position: Position) {
return comparePositions(range.start, position) >= 0 && return comparePositions(range.start, position) >= 0 && comparePositions(range.end, position) <= 0;
comparePositions(range.end, position) <= 0;
} }
export function rangesAreEqual(a: Range, b: Range) { export function rangesAreEqual(a: Range, b: Range) {

4
server/src/common/textRangeCollection.ts
View File

@ -107,9 +107,7 @@ export class TextRangeCollection<T extends TextRange> {
return mid; return mid;
} }
if (mid < this.count - 1 && TextRange.getEnd(item) <= position && if (mid < this.count - 1 && TextRange.getEnd(item) <= position && position < this._items[mid + 1].start) {
position < this._items[mid + 1].start) {
return -1; return -1;
} }

45
server/src/common/vfs.ts
View File

@ -15,7 +15,11 @@ import * as fs from 'fs';
import { ConsoleInterface, NullConsole } from './console'; import { ConsoleInterface, NullConsole } from './console';
export type Listener = (eventName: 'add' | 'addDir' | 'change' | 'unlink' | 'unlinkDir', path: string, stats?: Stats) => void; export type Listener = (
eventName: 'add' | 'addDir' | 'change' | 'unlink' | 'unlinkDir',
path: string,
stats?: Stats
) => void;
export interface FileWatcher { export interface FileWatcher {
close(): void; close(): void;
@ -61,21 +65,38 @@ const _isMacintosh = process.platform === 'darwin';
const _isLinux = process.platform === 'linux'; const _isLinux = process.platform === 'linux';
class FileSystem implements VirtualFileSystem { class FileSystem implements VirtualFileSystem {
constructor(private _console: ConsoleInterface) { constructor(private _console: ConsoleInterface) {}
}
existsSync(path: string) { return fs.existsSync(path); } existsSync(path: string) {
mkdirSync(path: string) { fs.mkdirSync(path); } return fs.existsSync(path);
chdir(path: string) { process.chdir(path); } }
readdirSync(path: string) { return fs.readdirSync(path); } mkdirSync(path: string) {
fs.mkdirSync(path);
}
chdir(path: string) {
process.chdir(path);
}
readdirSync(path: string) {
return fs.readdirSync(path);
}
readFileSync(path: string, encoding?: null): Buffer; readFileSync(path: string, encoding?: null): Buffer;
readFileSync(path: string, encoding: string): string; readFileSync(path: string, encoding: string): string;
readFileSync(path: string, encoding?: string | null): Buffer | string; readFileSync(path: string, encoding?: string | null): Buffer | string;
readFileSync(path: string, encoding: string | null = null) { return fs.readFileSync(path, { encoding }); } readFileSync(path: string, encoding: string | null = null) {
writeFileSync(path: string, data: string | Buffer, encoding: string | null) { fs.writeFileSync(path, data, { encoding }); } return fs.readFileSync(path, { encoding });
statSync(path: string) { return fs.statSync(path); } }
unlinkSync(path: string) { fs.unlinkSync(path); } writeFileSync(path: string, data: string | Buffer, encoding: string | null) {
realpathSync(path: string) { return fs.realpathSync(path); } fs.writeFileSync(path, data, { encoding });
}
statSync(path: string) {
return fs.statSync(path);
}
unlinkSync(path: string) {
fs.unlinkSync(path);
}
realpathSync(path: string) {
return fs.realpathSync(path);
}
getModulePath(): string { getModulePath(): string {
// The entry point to the tool should have set the __rootDirectory // The entry point to the tool should have set the __rootDirectory

126
server/src/languageServerBase.ts
View File

@ -7,11 +7,30 @@
import './common/extensions'; import './common/extensions';
import { import {
CodeAction, CodeActionKind, CodeActionParams, Command, CodeAction,
ConfigurationItem, createConnection, Diagnostic, DiagnosticRelatedInformation, CodeActionKind,
DiagnosticSeverity, DiagnosticTag, DocumentSymbol, ExecuteCommandParams, IConnection, CodeActionParams,
InitializeResult, Location, MarkupKind, ParameterInformation, RemoteConsole, RemoteWindow, SignatureInformation, Command,
SymbolInformation, TextDocuments, TextEdit, WorkspaceEdit ConfigurationItem,
createConnection,
Diagnostic,
DiagnosticRelatedInformation,
DiagnosticSeverity,
DiagnosticTag,
DocumentSymbol,
ExecuteCommandParams,
IConnection,
InitializeResult,
Location,
MarkupKind,
ParameterInformation,
RemoteConsole,
RemoteWindow,
SignatureInformation,
SymbolInformation,
TextDocuments,
TextEdit,
WorkspaceEdit
} from 'vscode-languageserver'; } from 'vscode-languageserver';
import { ImportResolver } from './analyzer/importResolver'; import { ImportResolver } from './analyzer/importResolver';
@ -98,7 +117,9 @@ export abstract class LanguageServerBase implements LanguageServerInterface {
} }
protected abstract async executeCommand(cmdParams: ExecuteCommandParams): Promise<any>; protected abstract async executeCommand(cmdParams: ExecuteCommandParams): Promise<any>;
protected abstract async executeCodeAction(cmdParams: CodeActionParams): Promise<(Command | CodeAction)[] | undefined | null>; protected abstract async executeCodeAction(
cmdParams: CodeActionParams
): Promise<(Command | CodeAction)[] | undefined | null>;
abstract async getSettings(workspace: WorkspaceServiceInstance): Promise<ServerSettings>; abstract async getSettings(workspace: WorkspaceServiceInstance): Promise<ServerSettings>;
protected getConfiguration(workspace: WorkspaceServiceInstance, section: string) { protected getConfiguration(workspace: WorkspaceServiceInstance, section: string) {
@ -156,8 +177,10 @@ export abstract class LanguageServerBase implements LanguageServerInterface {
} }
const fileOrFiles = results.filesRequiringAnalysis !== 1 ? 'files' : 'file'; const fileOrFiles = results.filesRequiringAnalysis !== 1 ? 'files' : 'file';
this._connection.sendNotification('pyright/reportProgress', this._connection.sendNotification(
`${ results.filesRequiringAnalysis } ${ fileOrFiles } to analyze`); 'pyright/reportProgress',
`${results.filesRequiringAnalysis} ${fileOrFiles} to analyze`
);
} }
} else { } else {
if (this._isDisplayingProgress) { if (this._isDisplayingProgress) {
@ -184,7 +207,8 @@ export abstract class LanguageServerBase implements LanguageServerInterface {
private _setupConnection(): void { private _setupConnection(): void {
// After the server has started the client sends an initialize request. The server receives // After the server has started the client sends an initialize request. The server receives
// in the passed params the rootPath of the workspace plus the client capabilities. // in the passed params the rootPath of the workspace plus the client capabilities.
this._connection.onInitialize((params): InitializeResult => { this._connection.onInitialize(
(params): InitializeResult => {
this.rootPath = params.rootPath || ''; this.rootPath = params.rootPath || '';
// Create a service instance for each of the workspace folders. // Create a service instance for each of the workspace folders.
@ -228,14 +252,12 @@ export abstract class LanguageServerBase implements LanguageServerInterface {
triggerCharacters: ['(', ',', ')'] triggerCharacters: ['(', ',', ')']
}, },
codeActionProvider: { codeActionProvider: {
codeActionKinds: [ codeActionKinds: [CodeActionKind.QuickFix, CodeActionKind.SourceOrganizeImports]
CodeActionKind.QuickFix,
CodeActionKind.SourceOrganizeImports
]
} }
} }
}; };
}); }
);
this._connection.onDidChangeConfiguration(_ => { this._connection.onDidChangeConfiguration(_ => {
this._connection.console.log(`Received updated settings`); this._connection.console.log(`Received updated settings`);
@ -262,8 +284,7 @@ export abstract class LanguageServerBase implements LanguageServerInterface {
if (!locations) { if (!locations) {
return undefined; return undefined;
} }
return locations.map(loc => return locations.map(loc => Location.create(convertPathToUri(loc.path), loc.range));
Location.create(convertPathToUri(loc.path), loc.range));
}); });
this._connection.onReferences(params => { this._connection.onReferences(params => {
@ -278,13 +299,15 @@ export abstract class LanguageServerBase implements LanguageServerInterface {
if (workspace.disableLanguageServices) { if (workspace.disableLanguageServices) {
return; return;
} }
const locations = workspace.serviceInstance.getReferencesForPosition(filePath, position, const locations = workspace.serviceInstance.getReferencesForPosition(
params.context.includeDeclaration); filePath,
position,
params.context.includeDeclaration
);
if (!locations) { if (!locations) {
return undefined; return undefined;
} }
return locations.map(loc => return locations.map(loc => Location.create(convertPathToUri(loc.path), loc.range));
Location.create(convertPathToUri(loc.path), loc.range));
}); });
this._connection.onDocumentSymbol(params => { this._connection.onDocumentSymbol(params => {
@ -307,8 +330,7 @@ export abstract class LanguageServerBase implements LanguageServerInterface {
this._workspaceMap.forEach(workspace => { this._workspaceMap.forEach(workspace => {
if (!workspace.disableLanguageServices) { if (!workspace.disableLanguageServices) {
workspace.serviceInstance.addSymbolsForWorkspace( workspace.serviceInstance.addSymbolsForWorkspace(symbolList, params.query);
symbolList, params.query);
} }
}); });
@ -329,12 +351,14 @@ export abstract class LanguageServerBase implements LanguageServerInterface {
return undefined; return undefined;
} }
const markupString = hoverResults.parts.map(part => { const markupString = hoverResults.parts
.map(part => {
if (part.python) { if (part.python) {
return '```python\n' + part.text + '\n```\n'; return '```python\n' + part.text + '\n```\n';
} }
return part.text; return part.text;
}).join(''); })
.join('');
return { return {
contents: { contents: {
@ -357,8 +381,7 @@ export abstract class LanguageServerBase implements LanguageServerInterface {
if (workspace.disableLanguageServices) { if (workspace.disableLanguageServices) {
return; return;
} }
const signatureHelpResults = workspace.serviceInstance.getSignatureHelpForPosition( const signatureHelpResults = workspace.serviceInstance.getSignatureHelpForPosition(filePath, position);
filePath, position);
if (!signatureHelpResults) { if (!signatureHelpResults) {
return undefined; return undefined;
} }
@ -369,16 +392,17 @@ export abstract class LanguageServerBase implements LanguageServerInterface {
if (sig.parameters) { if (sig.parameters) {
paramInfo = sig.parameters.map(param => { paramInfo = sig.parameters.map(param => {
return ParameterInformation.create( return ParameterInformation.create(
[param.startOffset, param.endOffset], param.documentation); [param.startOffset, param.endOffset],
param.documentation
);
}); });
} }
return SignatureInformation.create(sig.label, sig.documentation, return SignatureInformation.create(sig.label, sig.documentation, ...paramInfo);
...paramInfo);
}), }),
activeSignature: signatureHelpResults.activeSignature !== undefined ? activeSignature:
signatureHelpResults.activeSignature : null, signatureHelpResults.activeSignature !== undefined ? signatureHelpResults.activeSignature : null,
activeParameter: signatureHelpResults.activeParameter !== undefined ? activeParameter:
signatureHelpResults.activeParameter : null signatureHelpResults.activeParameter !== undefined ? signatureHelpResults.activeParameter : null
}; };
}); });
@ -396,7 +420,10 @@ export abstract class LanguageServerBase implements LanguageServerInterface {
} }
const completions = workspace.serviceInstance.getCompletionsForPosition( const completions = workspace.serviceInstance.getCompletionsForPosition(
filePath, position, workspace.rootPath); filePath,
position,
workspace.rootPath
);
// Always mark as incomplete so we get called back when the // Always mark as incomplete so we get called back when the
// user continues typing. Without this, the editor will assume // user continues typing. Without this, the editor will assume
@ -414,8 +441,7 @@ export abstract class LanguageServerBase implements LanguageServerInterface {
if (completionItemData) { if (completionItemData) {
const workspace = this._workspaceMap.get(completionItemData.workspacePath); const workspace = this._workspaceMap.get(completionItemData.workspacePath);
if (workspace && completionItemData.filePath) { if (workspace && completionItemData.filePath) {
workspace.serviceInstance.resolveCompletionItem( workspace.serviceInstance.resolveCompletionItem(completionItemData.filePath, params);
completionItemData.filePath, params);
} }
} }
return params; return params;
@ -433,8 +459,7 @@ export abstract class LanguageServerBase implements LanguageServerInterface {
if (workspace.disableLanguageServices) { if (workspace.disableLanguageServices) {
return; return;
} }
const editActions = workspace.serviceInstance.renameSymbolAtPosition( const editActions = workspace.serviceInstance.renameSymbolAtPosition(filePath, position, params.newName);
filePath, position, params.newName);
if (!editActions) { if (!editActions) {
return undefined; return undefined;
@ -462,10 +487,7 @@ export abstract class LanguageServerBase implements LanguageServerInterface {
this._connection.onDidOpenTextDocument(params => { this._connection.onDidOpenTextDocument(params => {
const filePath = convertUriToPath(params.textDocument.uri); const filePath = convertUriToPath(params.textDocument.uri);
const service = this._workspaceMap.getWorkspaceForFile(filePath).serviceInstance; const service = this._workspaceMap.getWorkspaceForFile(filePath).serviceInstance;
service.setFileOpened( service.setFileOpened(filePath, params.textDocument.version, params.textDocument.text);
filePath,
params.textDocument.version,
params.textDocument.text);
}); });
this._connection.onDidChangeTextDocument(params => { this._connection.onDidChangeTextDocument(params => {
@ -473,10 +495,7 @@ export abstract class LanguageServerBase implements LanguageServerInterface {
const filePath = convertUriToPath(params.textDocument.uri); const filePath = convertUriToPath(params.textDocument.uri);
const service = this._workspaceMap.getWorkspaceForFile(filePath).serviceInstance; const service = this._workspaceMap.getWorkspaceForFile(filePath).serviceInstance;
service.updateOpenFileContents( service.updateOpenFileContents(filePath, params.textDocument.version, params.contentChanges[0].text);
filePath,
params.textDocument.version,
params.contentChanges[0].text);
}); });
this._connection.onDidCloseTextDocument(params => { this._connection.onDidCloseTextDocument(params => {
@ -522,16 +541,18 @@ export abstract class LanguageServerBase implements LanguageServerInterface {
workspace.disableLanguageServices = !!serverSettings.disableLanguageServices; workspace.disableLanguageServices = !!serverSettings.disableLanguageServices;
} }
updateOptionsAndRestartService(workspace: WorkspaceServiceInstance, updateOptionsAndRestartService(
serverSettings: ServerSettings, typeStubTargetImportName?: string) { workspace: WorkspaceServiceInstance,
serverSettings: ServerSettings,
typeStubTargetImportName?: string
) {
AnalyzerServiceExecutor.runWithOptions(this.rootPath, workspace, serverSettings, typeStubTargetImportName); AnalyzerServiceExecutor.runWithOptions(this.rootPath, workspace, serverSettings, typeStubTargetImportName);
} }
private _convertDiagnostics(diags: AnalyzerDiagnostic[]): Diagnostic[] { private _convertDiagnostics(diags: AnalyzerDiagnostic[]): Diagnostic[] {
return diags.map(diag => { return diags.map(diag => {
const severity = diag.category === DiagnosticCategory.Error ? const severity =
DiagnosticSeverity.Error : DiagnosticSeverity.Warning; diag.category === DiagnosticCategory.Error ? DiagnosticSeverity.Error : DiagnosticSeverity.Warning;
let source = this._productName; let source = this._productName;
const rule = diag.getRule(); const rule = diag.getRule();
@ -539,8 +560,7 @@ export abstract class LanguageServerBase implements LanguageServerInterface {
source = `${source} (${rule})`; source = `${source} (${rule})`;
} }
const vsDiag = Diagnostic.create(diag.range, diag.message, severity, const vsDiag = Diagnostic.create(diag.range, diag.message, severity, undefined, source);
undefined, source);
if (diag.category === DiagnosticCategory.UnusedCode) { if (diag.category === DiagnosticCategory.UnusedCode) {
vsDiag.tags = [DiagnosticTag.Unnecessary]; vsDiag.tags = [DiagnosticTag.Unnecessary];

38
server/src/languageService/analyzerServiceExecutor.ts
View File

@ -12,20 +12,30 @@ import { combinePaths, normalizePath } from '../common/pathUtils';
import { ServerSettings, WorkspaceServiceInstance } from '../languageServerBase'; import { ServerSettings, WorkspaceServiceInstance } from '../languageServerBase';
export class AnalyzerServiceExecutor { export class AnalyzerServiceExecutor {
static runWithOptions(languageServiceRootPath: string, workspace: WorkspaceServiceInstance, static runWithOptions(
serverSettings: ServerSettings, typeStubTargetImportName?: string): void { languageServiceRootPath: string,
workspace: WorkspaceServiceInstance,
const commandLineOptions = GetCommandLineOptions(languageServiceRootPath, workspace.rootPath, serverSettings: ServerSettings,
serverSettings, typeStubTargetImportName); typeStubTargetImportName?: string
): void {
const commandLineOptions = GetCommandLineOptions(
languageServiceRootPath,
workspace.rootPath,
serverSettings,
typeStubTargetImportName
);
// setting option cause analyzer service to re-analyze everything // setting option cause analyzer service to re-analyze everything
workspace.serviceInstance.setOptions(commandLineOptions); workspace.serviceInstance.setOptions(commandLineOptions);
} }
} }
function GetCommandLineOptions(languageServiceRootPath: string, workspaceRootPath: string, function GetCommandLineOptions(
serverSettings: ServerSettings, typeStubTargetImportName?: string) { languageServiceRootPath: string,
workspaceRootPath: string,
serverSettings: ServerSettings,
typeStubTargetImportName?: string
) {
const commandLineOptions = new CommandLineOptions(workspaceRootPath, true); const commandLineOptions = new CommandLineOptions(workspaceRootPath, true);
commandLineOptions.checkOnlyOpenFiles = serverSettings.openFilesOnly; commandLineOptions.checkOnlyOpenFiles = serverSettings.openFilesOnly;
commandLineOptions.useLibraryCodeForTypes = serverSettings.useLibraryCodeForTypes; commandLineOptions.useLibraryCodeForTypes = serverSettings.useLibraryCodeForTypes;
@ -37,8 +47,10 @@ function GetCommandLineOptions(languageServiceRootPath: string, workspaceRootPat
commandLineOptions.watch = !commandLineOptions.checkOnlyOpenFiles; commandLineOptions.watch = !commandLineOptions.checkOnlyOpenFiles;
if (serverSettings.venvPath) { if (serverSettings.venvPath) {
commandLineOptions.venvPath = combinePaths(workspaceRootPath || languageServiceRootPath, commandLineOptions.venvPath = combinePaths(
normalizePath(_expandPathVariables(languageServiceRootPath, serverSettings.venvPath))); workspaceRootPath || languageServiceRootPath,
normalizePath(_expandPathVariables(languageServiceRootPath, serverSettings.venvPath))
);
} }
if (serverSettings.pythonPath) { if (serverSettings.pythonPath) {
@ -46,8 +58,10 @@ function GetCommandLineOptions(languageServiceRootPath: string, workspaceRootPat
// the local python interpreter should be used rather than interpreting the // the local python interpreter should be used rather than interpreting the
// setting value as a path to the interpreter. We'll simply ignore it in this case. // setting value as a path to the interpreter. We'll simply ignore it in this case.
if (serverSettings.pythonPath.trim() !== 'python') { if (serverSettings.pythonPath.trim() !== 'python') {
commandLineOptions.pythonPath = combinePaths(workspaceRootPath || languageServiceRootPath, commandLineOptions.pythonPath = combinePaths(
normalizePath(_expandPathVariables(languageServiceRootPath, serverSettings.pythonPath))); workspaceRootPath || languageServiceRootPath,
normalizePath(_expandPathVariables(languageServiceRootPath, serverSettings.pythonPath))
);
} }
} }

40
server/src/languageService/codeActionProvider.ts
View File

@ -14,8 +14,10 @@ import { WorkspaceServiceInstance } from '../languageServerBase';
export class CodeActionProvider { export class CodeActionProvider {
static getCodeActionsForPosition(workspace: WorkspaceServiceInstance, filePath: string, range: Range) { static getCodeActionsForPosition(workspace: WorkspaceServiceInstance, filePath: string, range: Range) {
const sortImportsCodeAction = CodeAction.create( const sortImportsCodeAction = CodeAction.create(
'Organize Imports', Command.create('Organize Imports', Commands.orderImports), 'Organize Imports',
CodeActionKind.SourceOrganizeImports); Command.create('Organize Imports', Commands.orderImports),
CodeActionKind.SourceOrganizeImports
);
const codeActions: CodeAction[] = [sortImportsCodeAction]; const codeActions: CodeAction[] = [sortImportsCodeAction];
if (!workspace.disableLanguageServices) { if (!workspace.disableLanguageServices) {
@ -26,14 +28,21 @@ export class CodeActionProvider {
}); });
if (typeStubDiag) { if (typeStubDiag) {
const action = typeStubDiag.getActions()!.find( const action = typeStubDiag
a => a.action === Commands.createTypeStub) as CreateTypeStubFileAction; .getActions()!
.find(a => a.action === Commands.createTypeStub) as CreateTypeStubFileAction;
if (action) { if (action) {
const createTypeStubAction = CodeAction.create( const createTypeStubAction = CodeAction.create(
`Create Type Stub For ‘${ action.moduleName }’`, `Create Type Stub For '${action.moduleName}'`,
Command.create('Create Type Stub', Commands.createTypeStub, Command.create(
workspace.rootPath, action.moduleName, filePath), 'Create Type Stub',
CodeActionKind.QuickFix); Commands.createTypeStub,
workspace.rootPath,
action.moduleName,
filePath
),
CodeActionKind.QuickFix
);
codeActions.push(createTypeStubAction); codeActions.push(createTypeStubAction);
} }
} }
@ -44,14 +53,19 @@ export class CodeActionProvider {
}); });
if (addOptionalDiag) { if (addOptionalDiag) {
const action = addOptionalDiag.getActions()!.find( const action = addOptionalDiag
a => a.action === Commands.addMissingOptionalToParam) as AddMissingOptionalToParamAction; .getActions()!
.find(a => a.action === Commands.addMissingOptionalToParam) as AddMissingOptionalToParamAction;
if (action) { if (action) {
const addMissingOptionalAction = CodeAction.create( const addMissingOptionalAction = CodeAction.create(
`Add 'Optional' to type annotation`, `Add 'Optional' to type annotation`,
Command.create(`Add 'Optional' to type annotation`, Commands.addMissingOptionalToParam, Command.create(
action.offsetOfTypeNode), `Add 'Optional' to type annotation`,
CodeActionKind.QuickFix); Commands.addMissingOptionalToParam,
action.offsetOfTypeNode
),
CodeActionKind.QuickFix
);
codeActions.push(addMissingOptionalAction); codeActions.push(addMissingOptionalAction);
} }
} }

436
server/src/languageService/completionProvider.ts
View File

@ -8,8 +8,7 @@
* a list of zero or more text completions that apply in the context. * a list of zero or more text completions that apply in the context.
*/ */
import { CompletionItem, CompletionItemKind, CompletionList, import { CompletionItem, CompletionItemKind, CompletionList, MarkupKind, Range, TextEdit } from 'vscode-languageserver';
MarkupKind, Range, TextEdit } from 'vscode-languageserver';
import { ImportLookup } from '../analyzer/analyzerFileInfo'; import { ImportLookup } from '../analyzer/analyzerFileInfo';
import * as AnalyzerNodeInfo from '../analyzer/analyzerNodeInfo'; import * as AnalyzerNodeInfo from '../analyzer/analyzerNodeInfo';
@ -32,9 +31,20 @@ import { convertOffsetToPosition, convertPositionToOffset } from '../common/posi
import * as StringUtils from '../common/stringUtils'; import * as StringUtils from '../common/stringUtils';
import { comparePositions, Position } from '../common/textRange'; import { comparePositions, Position } from '../common/textRange';
import { TextRange } from '../common/textRange'; import { TextRange } from '../common/textRange';
import { ErrorExpressionCategory, ErrorNode, ExpressionNode, import {
FunctionNode, ImportFromNode, isExpressionNode, ModuleNameNode, NameNode, ErrorExpressionCategory,
ParameterCategory, ParseNode, ParseNodeType, StringNode } from '../parser/parseNodes'; ErrorNode,
ExpressionNode,
FunctionNode,
ImportFromNode,
isExpressionNode,
ModuleNameNode,
NameNode,
ParameterCategory,
ParseNode,
ParseNodeType,
StringNode
} from '../parser/parseNodes';
import { ParseResults } from '../parser/parser'; import { ParseResults } from '../parser/parser';
const _keywords: string[] = [ const _keywords: string[] = [
@ -161,8 +171,8 @@ export class CompletionProvider {
private _configOptions: ConfigOptions, private _configOptions: ConfigOptions,
private _importLookup: ImportLookup, private _importLookup: ImportLookup,
private _evaluator: TypeEvaluator, private _evaluator: TypeEvaluator,
private _moduleSymbolsCallback: () => ModuleSymbolMap) { private _moduleSymbolsCallback: () => ModuleSymbolMap
} ) {}
getCompletionsForPosition(): CompletionList | undefined { getCompletionsForPosition(): CompletionList | undefined {
const offset = convertPositionToOffset(this._position, this._parseResults.tokenizerOutput.lines); const offset = convertPositionToOffset(this._position, this._parseResults.tokenizerOutput.lines);
@ -244,8 +254,7 @@ export class CompletionProvider {
} }
if (curNode.nodeType === ParseNodeType.Error) { if (curNode.nodeType === ParseNodeType.Error) {
return this._getExpressionErrorCompletions(curNode, priorWord, return this._getExpressionErrorCompletions(curNode, priorWord, priorText, postText);
priorText, postText);
} }
if (curNode.nodeType === ParseNodeType.MemberAccess) { if (curNode.nodeType === ParseNodeType.MemberAccess) {
@ -268,8 +277,7 @@ export class CompletionProvider {
} }
} }
} else if (curNode.parent && curNode.parent.nodeType === ParseNodeType.MemberAccess) { } else if (curNode.parent && curNode.parent.nodeType === ParseNodeType.MemberAccess) {
return this._getMemberAccessCompletions( return this._getMemberAccessCompletions(curNode.parent.leftExpression, priorWord);
curNode.parent.leftExpression, priorWord);
} }
} }
@ -308,8 +316,8 @@ export class CompletionProvider {
} }
const curIndex = CompletionProvider._mostRecentCompletions.findIndex( const curIndex = CompletionProvider._mostRecentCompletions.findIndex(
item => item.label === label && item => item.label === label && item.autoImportText === autoImportText
item.autoImportText === autoImportText); );
if (curIndex > 0) { if (curIndex > 0) {
// If there's an existing entry with the same name that's not at the // If there's an existing entry with the same name that's not at the
@ -356,9 +364,12 @@ export class CompletionProvider {
return !!priorText.match(/#/); return !!priorText.match(/#/);
} }
private _getExpressionErrorCompletions(node: ErrorNode, priorWord: string, private _getExpressionErrorCompletions(
priorText: string, postText: string): CompletionList | undefined { node: ErrorNode,
priorWord: string,
priorText: string,
postText: string
): CompletionList | undefined {
// Is the error due to a missing member access name? If so, // Is the error due to a missing member access name? If so,
// we can evaluate the left side of the member access expression // we can evaluate the left side of the member access expression
// to determine its type and offer suggestions based on it. // to determine its type and offer suggestions based on it.
@ -400,8 +411,7 @@ export class CompletionProvider {
private _createSingleKeywordCompletionList(keyword: string): CompletionList { private _createSingleKeywordCompletionList(keyword: string): CompletionList {
const completionItem = CompletionItem.create(keyword); const completionItem = CompletionItem.create(keyword);
completionItem.kind = CompletionItemKind.Keyword; completionItem.kind = CompletionItemKind.Keyword;
completionItem.sortText = completionItem.sortText = this._makeSortText(SortCategory.LikelyKeyword, keyword);
this._makeSortText(SortCategory.LikelyKeyword, keyword);
return CompletionList.create([completionItem]); return CompletionList.create([completionItem]);
} }
@ -439,8 +449,7 @@ export class CompletionProvider {
const methodSignature = this._printMethodSignature(decl.node) + ':'; const methodSignature = this._printMethodSignature(decl.node) + ':';
const textEdit = TextEdit.replace(range, methodSignature); const textEdit = TextEdit.replace(range, methodSignature);
this._addSymbol(name, symbol, partialName.value, completionList, this._addSymbol(name, symbol, partialName.value, completionList, undefined, textEdit);
undefined, textEdit);
} }
} }
}); });
@ -449,7 +458,8 @@ export class CompletionProvider {
} }
private _printMethodSignature(node: FunctionNode): string { private _printMethodSignature(node: FunctionNode): string {
const paramList = node.parameters.map(param => { const paramList = node.parameters
.map(param => {
let paramString = ''; let paramString = '';
if (param.category === ParameterCategory.VarArgList) { if (param.category === ParameterCategory.VarArgList) {
paramString += '*'; paramString += '*';
@ -466,7 +476,8 @@ export class CompletionProvider {
} }
return paramString; return paramString;
}).join(', '); })
.join(', ');
let methodSignature = node.name.value + '(' + paramList + ')'; let methodSignature = node.name.value + '(' + paramList + ')';
@ -477,9 +488,7 @@ export class CompletionProvider {
return methodSignature; return methodSignature;
} }
private _getMemberAccessCompletions(leftExprNode: ExpressionNode, private _getMemberAccessCompletions(leftExprNode: ExpressionNode, priorWord: string): CompletionList | undefined {
priorWord: string): CompletionList | undefined {
const leftType = this._evaluator.getType(leftExprNode); const leftType = this._evaluator.getType(leftExprNode);
const symbolTable = new Map<string, Symbol>(); const symbolTable = new Map<string, Symbol>();
@ -498,23 +507,27 @@ export class CompletionProvider {
} }
const completionList = CompletionList.create(); const completionList = CompletionList.create();
this._addSymbolsForSymbolTable(symbolTable, _ => true, this._addSymbolsForSymbolTable(symbolTable, _ => true, priorWord, completionList);
priorWord, completionList);
return completionList; return completionList;
} }
private _getStatementCompletions(parseNode: ParseNode, priorWord: string, private _getStatementCompletions(
priorText: string, postText: string): CompletionList | undefined { parseNode: ParseNode,
priorWord: string,
priorText: string,
postText: string
): CompletionList | undefined {
// For now, use the same logic for expressions and statements. // For now, use the same logic for expressions and statements.
return this._getExpressionCompletions(parseNode, priorWord, return this._getExpressionCompletions(parseNode, priorWord, priorText, postText);
priorText, postText);
} }
private _getExpressionCompletions(parseNode: ParseNode, priorWord: string, private _getExpressionCompletions(
priorText: string, postText: string): CompletionList | undefined { parseNode: ParseNode,
priorWord: string,
priorText: string,
postText: string
): CompletionList | undefined {
// If the user typed a "." as part of a number, don't present // If the user typed a "." as part of a number, don't present
// any completion options. // any completion options.
if (parseNode.nodeType === ParseNodeType.Number) { if (parseNode.nodeType === ParseNodeType.Number) {
@ -534,8 +547,7 @@ export class CompletionProvider {
const completionItem = CompletionItem.create(keyword); const completionItem = CompletionItem.create(keyword);
completionItem.kind = CompletionItemKind.Keyword; completionItem.kind = CompletionItemKind.Keyword;
completionList.items.push(completionItem); completionList.items.push(completionItem);
completionItem.sortText = completionItem.sortText = this._makeSortText(SortCategory.Keyword, keyword);
this._makeSortText(SortCategory.Keyword, keyword);
}); });
// Add auto-import suggestions from other modules. // Add auto-import suggestions from other modules.
@ -551,11 +563,11 @@ export class CompletionProvider {
} else if (parseNode.category === ErrorExpressionCategory.MissingExpression) { } else if (parseNode.category === ErrorExpressionCategory.MissingExpression) {
if (parseNode.parent && parseNode.parent.nodeType === ParseNodeType.Assignment) { if (parseNode.parent && parseNode.parent.nodeType === ParseNodeType.Assignment) {
const declaredTypeOfTarget = this._evaluator.getDeclaredTypeForExpression( const declaredTypeOfTarget = this._evaluator.getDeclaredTypeForExpression(
parseNode.parent.leftExpression); parseNode.parent.leftExpression
);
if (declaredTypeOfTarget) { if (declaredTypeOfTarget) {
this._addLiteralValuesForTargetType(declaredTypeOfTarget, this._addLiteralValuesForTargetType(declaredTypeOfTarget, priorText, postText, completionList);
priorText, postText, completionList);
} }
} }
} }
@ -564,20 +576,23 @@ export class CompletionProvider {
return completionList; return completionList;
} }
private _addCallArgumentCompletions(parseNode: ParseNode, priorWord: string, priorText: string, private _addCallArgumentCompletions(
postText: string, completionList: CompletionList) { parseNode: ParseNode,
priorWord: string,
priorText: string,
postText: string,
completionList: CompletionList
) {
// If we're within the argument list of a call, add parameter names. // If we're within the argument list of a call, add parameter names.
const offset = convertPositionToOffset(this._position, const offset = convertPositionToOffset(this._position, this._parseResults.tokenizerOutput.lines)!;
this._parseResults.tokenizerOutput.lines)!; const signatureInfo = this._evaluator.getCallSignatureInfo(parseNode, offset);
const signatureInfo = this._evaluator.getCallSignatureInfo(parseNode, offset)
if (signatureInfo) { if (signatureInfo) {
// Are we past the call expression and within the argument list? // Are we past the call expression and within the argument list?
const callNameEnd = convertOffsetToPosition( const callNameEnd = convertOffsetToPosition(
signatureInfo.callNode.leftExpression.start + signatureInfo.callNode.leftExpression.start + signatureInfo.callNode.leftExpression.length,
signatureInfo.callNode.leftExpression.length, this._parseResults.tokenizerOutput.lines
this._parseResults.tokenizerOutput.lines); );
if (comparePositions(this._position, callNameEnd) > 0) { if (comparePositions(this._position, callNameEnd) > 0) {
this._addNamedParameters(signatureInfo, priorWord, completionList); this._addNamedParameters(signatureInfo, priorWord, completionList);
@ -588,15 +603,19 @@ export class CompletionProvider {
} }
} }
private _addLiteralValuesForArgument(signatureInfo: CallSignatureInfo, private _addLiteralValuesForArgument(
priorText: string, postText: string, completionList: CompletionList) { signatureInfo: CallSignatureInfo,
priorText: string,
postText: string,
completionList: CompletionList
) {
signatureInfo.signatures.forEach(signature => { signatureInfo.signatures.forEach(signature => {
let paramIndex = -1; let paramIndex = -1;
if (signatureInfo.activeArgumentName !== undefined) { if (signatureInfo.activeArgumentName !== undefined) {
paramIndex = signature.details.parameters.findIndex(param => { paramIndex = signature.details.parameters.findIndex(param => {
param.name === signatureInfo.activeArgumentName}); param.name === signatureInfo.activeArgumentName;
});
} else if (signatureInfo.activeArgumentIndex < signature.details.parameters.length) { } else if (signatureInfo.activeArgumentIndex < signature.details.parameters.length) {
paramIndex = signatureInfo.activeArgumentIndex; paramIndex = signatureInfo.activeArgumentIndex;
} }
@ -611,17 +630,24 @@ export class CompletionProvider {
}); });
} }
private _addLiteralValuesForTargetType(type: Type, priorText: string, postText: string, private _addLiteralValuesForTargetType(
completionList: CompletionList) { type: Type,
priorText: string,
postText: string,
completionList: CompletionList
) {
const quoteValue = this._getQuoteValueFromPriorText(priorText); const quoteValue = this._getQuoteValueFromPriorText(priorText);
doForSubtypes(type, subtype => { doForSubtypes(type, subtype => {
if (subtype.category === TypeCategory.Object) { if (subtype.category === TypeCategory.Object) {
if (ClassType.isBuiltIn(subtype.classType, 'str')) { if (ClassType.isBuiltIn(subtype.classType, 'str')) {
if (subtype.literalValue !== undefined) { if (subtype.literalValue !== undefined) {
this._addStringLiteralToCompletionList(subtype.literalValue as string, this._addStringLiteralToCompletionList(
quoteValue.stringValue, postText, quoteValue.quoteCharacter, subtype.literalValue as string,
completionList); quoteValue.stringValue,
postText,
quoteValue.quoteCharacter,
completionList
);
} }
} }
} }
@ -630,9 +656,12 @@ export class CompletionProvider {
}); });
} }
private _getStringLiteralCompletions(parseNode: StringNode, priorWord: string, private _getStringLiteralCompletions(
priorText: string, postText: string): CompletionList | undefined { parseNode: StringNode,
priorWord: string,
priorText: string,
postText: string
): CompletionList | undefined {
let parentNode: ParseNode | undefined = parseNode.parent; let parentNode: ParseNode | undefined = parseNode.parent;
if (!parentNode || parentNode.nodeType !== ParseNodeType.StringList || parentNode.strings.length > 1) { if (!parentNode || parentNode.nodeType !== ParseNodeType.StringList || parentNode.strings.length > 1) {
return undefined; return undefined;
@ -666,16 +695,19 @@ export class CompletionProvider {
const quoteValue = this._getQuoteValueFromPriorText(priorText); const quoteValue = this._getQuoteValueFromPriorText(priorText);
entries.forEach((_, key) => { entries.forEach((_, key) => {
this._addStringLiteralToCompletionList(key, quoteValue.stringValue, postText, this._addStringLiteralToCompletionList(
quoteValue.quoteCharacter, completionList); key,
quoteValue.stringValue,
postText,
quoteValue.quoteCharacter,
completionList
);
}); });
} else if (parentNode.nodeType === ParseNodeType.Assignment) { } else if (parentNode.nodeType === ParseNodeType.Assignment) {
const declaredTypeOfTarget = this._evaluator.getDeclaredTypeForExpression( const declaredTypeOfTarget = this._evaluator.getDeclaredTypeForExpression(parentNode.leftExpression);
parentNode.leftExpression);
if (declaredTypeOfTarget) { if (declaredTypeOfTarget) {
this._addLiteralValuesForTargetType(declaredTypeOfTarget, this._addLiteralValuesForTargetType(declaredTypeOfTarget, priorText, postText, completionList);
priorText, postText, completionList);
} }
} else { } else {
this._addCallArgumentCompletions(parseNode, priorWord, priorText, postText, completionList); this._addCallArgumentCompletions(parseNode, priorWord, priorText, postText, completionList);
@ -689,14 +721,14 @@ export class CompletionProvider {
// (either starting with a single or double quote). Returns the quote // (either starting with a single or double quote). Returns the quote
// type and the string literal value after the starting quote. // type and the string literal value after the starting quote.
private _getQuoteValueFromPriorText(priorText: string) { private _getQuoteValueFromPriorText(priorText: string) {
const lastSingleQuote = priorText.lastIndexOf('\''); const lastSingleQuote = priorText.lastIndexOf("'");
const lastDoubleQuote = priorText.lastIndexOf('"'); const lastDoubleQuote = priorText.lastIndexOf('"');
let quoteCharacter = this._parseResults.tokenizerOutput.predominantSingleQuoteCharacter; let quoteCharacter = this._parseResults.tokenizerOutput.predominantSingleQuoteCharacter;
let stringValue = undefined; let stringValue = undefined;
if (lastSingleQuote > lastDoubleQuote) { if (lastSingleQuote > lastDoubleQuote) {
quoteCharacter = '\''; quoteCharacter = "'";
stringValue = priorText.substr(lastSingleQuote + 1); stringValue = priorText.substr(lastSingleQuote + 1);
} else if (lastDoubleQuote > lastSingleQuote) { } else if (lastDoubleQuote > lastSingleQuote) {
quoteCharacter = '"'; quoteCharacter = '"';
@ -729,15 +761,23 @@ export class CompletionProvider {
const entries = this._evaluator.getTypedDictMembersForClass(classType); const entries = this._evaluator.getTypedDictMembersForClass(classType);
entries.forEach((_, key) => { entries.forEach((_, key) => {
this._addStringLiteralToCompletionList(key, undefined, undefined, this._addStringLiteralToCompletionList(
key,
undefined,
undefined,
this._parseResults.tokenizerOutput.predominantSingleQuoteCharacter, this._parseResults.tokenizerOutput.predominantSingleQuoteCharacter,
completionList); completionList
);
}); });
} }
private _addStringLiteralToCompletionList(value: string, priorString: string | undefined, private _addStringLiteralToCompletionList(
postText: string | undefined, quoteCharacter: string, completionList: CompletionList) { value: string,
priorString: string | undefined,
postText: string | undefined,
quoteCharacter: string,
completionList: CompletionList
) {
const isSimilar = StringUtils.computeCompletionSimilarity(priorString || '', value) > similarityLimit; const isSimilar = StringUtils.computeCompletionSimilarity(priorString || '', value) > similarityLimit;
if (isSimilar) { if (isSimilar) {
const valueWithQuotes = `${quoteCharacter}${value}${quoteCharacter}`; const valueWithQuotes = `${quoteCharacter}${value}${quoteCharacter}`;
@ -771,8 +811,7 @@ export class CompletionProvider {
private _getAutoImportCompletions(priorWord: string, completionList: CompletionList) { private _getAutoImportCompletions(priorWord: string, completionList: CompletionList) {
const moduleSymbolMap = this._moduleSymbolsCallback(); const moduleSymbolMap = this._moduleSymbolsCallback();
const importStatements = ImportStatementUtils.getTopLevelImports( const importStatements = ImportStatementUtils.getTopLevelImports(this._parseResults.parseTree);
this._parseResults.parseTree);
moduleSymbolMap.forEach((symbolTable, filePath) => { moduleSymbolMap.forEach((symbolTable, filePath) => {
const fileName = stripFileExtension(getFileName(filePath)); const fileName = stripFileExtension(getFileName(filePath));
@ -784,9 +823,10 @@ export class CompletionProvider {
// For very short matching strings, we will require an exact match. Otherwise // For very short matching strings, we will require an exact match. Otherwise
// we will tend to return a list that's too long. Once we get beyond two // we will tend to return a list that's too long. Once we get beyond two
// characters, we can do a fuzzy match. // characters, we can do a fuzzy match.
const isSimilar = priorWord.length > 2 ? const isSimilar =
StringUtils.computeCompletionSimilarity(priorWord, name) > similarityLimit : priorWord.length > 2
priorWord.length > 0 && name.startsWith(priorWord); ? StringUtils.computeCompletionSimilarity(priorWord, name) > similarityLimit
: priorWord.length > 0 && name.startsWith(priorWord);
if (isSimilar) { if (isSimilar) {
if (!symbol.isExternallyHidden()) { if (!symbol.isExternallyHidden()) {
@ -794,7 +834,8 @@ export class CompletionProvider {
// this name, don't add an auto-import suggestion with // this name, don't add an auto-import suggestion with
// the same name. // the same name.
const localDuplicate = completionList.items.find( const localDuplicate = completionList.items.find(
item => item.label === name && !item.data.autoImport); item => item.label === name && !item.data.autoImport
);
const declarations = symbol.getDeclarations(); const declarations = symbol.getDeclarations();
if (declarations && declarations.length > 0 && localDuplicate === undefined) { if (declarations && declarations.length > 0 && localDuplicate === undefined) {
// Don't include imported symbols, only those that // Don't include imported symbols, only those that
@ -812,11 +853,22 @@ export class CompletionProvider {
} }
const autoImportTextEdits = this._getTextEditsForAutoImportByFilePath( const autoImportTextEdits = this._getTextEditsForAutoImportByFilePath(
name, importStatements, filePath, importSource, name,
moduleNameAndType ? moduleNameAndType.importType : ImportType.Local); importStatements,
filePath,
importSource,
moduleNameAndType ? moduleNameAndType.importType : ImportType.Local
);
this._addSymbol(name, symbol, priorWord, this._addSymbol(
completionList, importSource, undefined, autoImportTextEdits); name,
symbol,
priorWord,
completionList,
importSource,
undefined,
autoImportTextEdits
);
} }
} }
} }
@ -832,8 +884,7 @@ export class CompletionProvider {
// file, we can use that directory name as an implicit import target. // file, we can use that directory name as an implicit import target.
if (moduleSymbolMap.has(initPathPy) || moduleSymbolMap.has(initPathPyi)) { if (moduleSymbolMap.has(initPathPy) || moduleSymbolMap.has(initPathPyi)) {
const name = getFileName(fileDir); const name = getFileName(fileDir);
const moduleNameAndType = this._getModuleNameAndTypeFromFilePath( const moduleNameAndType = this._getModuleNameAndTypeFromFilePath(getDirectoryPath(fileDir));
getDirectoryPath(fileDir));
if (moduleNameAndType.moduleName) { if (moduleNameAndType.moduleName) {
const autoImportText = `Auto-import from ${moduleNameAndType.moduleName}`; const autoImportText = `Auto-import from ${moduleNameAndType.moduleName}`;
@ -855,10 +906,23 @@ export class CompletionProvider {
if (!isDuplicateEntry) { if (!isDuplicateEntry) {
const autoImportTextEdits = this._getTextEditsForAutoImportByFilePath( const autoImportTextEdits = this._getTextEditsForAutoImportByFilePath(
name, importStatements, filePath, moduleNameAndType.moduleName, name,
moduleNameAndType ? moduleNameAndType.importType : ImportType.Local); importStatements,
this._addNameToCompletionList(name, CompletionItemKind.Module, priorWord, completionList, filePath,
name, '', autoImportText, undefined, autoImportTextEdits); moduleNameAndType.moduleName,
moduleNameAndType ? moduleNameAndType.importType : ImportType.Local
);
this._addNameToCompletionList(
name,
CompletionItemKind.Module,
priorWord,
completionList,
name,
'',
autoImportText,
undefined,
autoImportTextEdits
);
} }
} }
} }
@ -871,28 +935,36 @@ export class CompletionProvider {
// 'import from' statement. // 'import from' statement.
private _getModuleNameAndTypeFromFilePath(filePath: string): ModuleNameAndType { private _getModuleNameAndTypeFromFilePath(filePath: string): ModuleNameAndType {
const execEnvironment = this._configOptions.findExecEnvironment(this._filePath); const execEnvironment = this._configOptions.findExecEnvironment(this._filePath);
return this._importResolver.getModuleNameForImport( return this._importResolver.getModuleNameForImport(filePath, execEnvironment);
filePath, execEnvironment);
} }
private _getTextEditsForAutoImportByFilePath(symbolName: string, private _getTextEditsForAutoImportByFilePath(
importStatements: ImportStatementUtils.ImportStatements, filePath: string, symbolName: string,
moduleName: string, importType: ImportType): TextEditAction[] { importStatements: ImportStatementUtils.ImportStatements,
filePath: string,
moduleName: string,
importType: ImportType
): TextEditAction[] {
// Does an 'import from' statement already exist? If so, we'll reuse it. // Does an 'import from' statement already exist? If so, we'll reuse it.
const importStatement = importStatements.mapByFilePath.get(filePath); const importStatement = importStatements.mapByFilePath.get(filePath);
if (importStatement && importStatement.node.nodeType === ParseNodeType.ImportFrom) { if (importStatement && importStatement.node.nodeType === ParseNodeType.ImportFrom) {
return ImportStatementUtils.getTextEditsForAutoImportSymbolAddition( return ImportStatementUtils.getTextEditsForAutoImportSymbolAddition(
symbolName, importStatement, this._parseResults); symbolName,
importStatement,
this._parseResults
);
} }
return ImportStatementUtils.getTextEditsForAutoImportInsertion(symbolName, return ImportStatementUtils.getTextEditsForAutoImportInsertion(
importStatements, moduleName, importType, this._parseResults); symbolName,
importStatements,
moduleName,
importType,
this._parseResults
);
} }
private _getImportFromCompletions(importFromNode: ImportFromNode, private _getImportFromCompletions(importFromNode: ImportFromNode, priorWord: string): CompletionList | undefined {
priorWord: string): CompletionList | undefined {
// Don't attempt to provide completions for "from X import *". // Don't attempt to provide completions for "from X import *".
if (importFromNode.isWildcardImport) { if (importFromNode.isWildcardImport) {
return undefined; return undefined;
@ -907,25 +979,26 @@ export class CompletionProvider {
const completionList = CompletionList.create(); const completionList = CompletionList.create();
const resolvedPath = importInfo.resolvedPaths.length > 0 ? const resolvedPath =
importInfo.resolvedPaths[importInfo.resolvedPaths.length - 1] : ''; importInfo.resolvedPaths.length > 0 ? importInfo.resolvedPaths[importInfo.resolvedPaths.length - 1] : '';
const lookupResults = this._importLookup(resolvedPath); const lookupResults = this._importLookup(resolvedPath);
if (lookupResults) { if (lookupResults) {
this._addSymbolsForSymbolTable(lookupResults.symbolTable, this._addSymbolsForSymbolTable(
lookupResults.symbolTable,
name => { name => {
// Don't suggest symbols that have already been imported. // Don't suggest symbols that have already been imported.
return !importFromNode.imports.find( return !importFromNode.imports.find(imp => imp.name.value === name);
imp => imp.name.value === name);
}, },
priorWord, completionList); priorWord,
completionList
);
} }
// Add the implicit imports. // Add the implicit imports.
importInfo.implicitImports.forEach(implImport => { importInfo.implicitImports.forEach(implImport => {
if (!importFromNode.imports.find(imp => imp.name.value === implImport.name)) { if (!importFromNode.imports.find(imp => imp.name.value === implImport.name)) {
this._addNameToCompletionList(implImport.name, CompletionItemKind.Module, this._addNameToCompletionList(implImport.name, CompletionItemKind.Module, priorWord, completionList);
priorWord, completionList);
} }
}); });
@ -997,8 +1070,7 @@ export class CompletionProvider {
let scope = AnalyzerNodeInfo.getScope(curNode); let scope = AnalyzerNodeInfo.getScope(curNode);
if (scope) { if (scope) {
while (scope) { while (scope) {
this._addSymbolsForSymbolTable(scope.symbolTable, this._addSymbolsForSymbolTable(scope.symbolTable, () => true, priorWord, completionList);
() => true, priorWord, completionList);
scope = scope.parent; scope = scope.parent;
} }
break; break;
@ -1008,10 +1080,12 @@ export class CompletionProvider {
} }
} }
private _addSymbolsForSymbolTable(symbolTable: SymbolTable, private _addSymbolsForSymbolTable(
includeSymbolCallback: (name: string) => boolean, priorWord: string, symbolTable: SymbolTable,
completionList: CompletionList) { includeSymbolCallback: (name: string) => boolean,
priorWord: string,
completionList: CompletionList
) {
symbolTable.forEach((symbol, name) => { symbolTable.forEach((symbol, name) => {
// If there are no declarations or the symbol is not // If there are no declarations or the symbol is not
// exported from this scope, don't include it in the // exported from this scope, don't include it in the
@ -1022,9 +1096,15 @@ export class CompletionProvider {
}); });
} }
private _addSymbol(name: string, symbol: Symbol, priorWord: string, completionList: CompletionList, private _addSymbol(
autoImportSource?: string, textEdit?: TextEdit, additionalTextEdits?: TextEditAction[]) { name: string,
symbol: Symbol,
priorWord: string,
completionList: CompletionList,
autoImportSource?: string,
textEdit?: TextEdit,
additionalTextEdits?: TextEditAction[]
) {
let primaryDecl = getLastTypedDeclaredForSymbol(symbol); let primaryDecl = getLastTypedDeclaredForSymbol(symbol);
if (!primaryDecl) { if (!primaryDecl) {
const declarations = symbol.getDeclarations(); const declarations = symbol.getDeclarations();
@ -1063,8 +1143,9 @@ export class CompletionProvider {
case DeclarationType.Function: case DeclarationType.Function:
if (type.category === TypeCategory.OverloadedFunction) { if (type.category === TypeCategory.OverloadedFunction) {
typeDetail = type.overloads.map(overload => typeDetail = type.overloads
name + this._evaluator.printType(overload)).join('\n'); .map(overload => name + this._evaluator.printType(overload))
.join('\n');
} else { } else {
typeDetail = name + ': ' + this._evaluator.printType(type); typeDetail = name + ': ' + this._evaluator.printType(type);
} }
@ -1124,26 +1205,51 @@ export class CompletionProvider {
autoImportText = `Auto-import from ${autoImportSource}`; autoImportText = `Auto-import from ${autoImportSource}`;
} }
this._addNameToCompletionList(name, itemKind, priorWord, completionList, this._addNameToCompletionList(
undefined, undefined, autoImportText, textEdit, name,
additionalTextEdits, symbol.id); itemKind,
priorWord,
completionList,
undefined,
undefined,
autoImportText,
textEdit,
additionalTextEdits,
symbol.id
);
} else { } else {
// Does the symbol have no declaration but instead has a synthesized type? // Does the symbol have no declaration but instead has a synthesized type?
const synthesizedType = symbol.getSynthesizedType(); const synthesizedType = symbol.getSynthesizedType();
if (synthesizedType) { if (synthesizedType) {
const itemKind: CompletionItemKind = CompletionItemKind.Variable; const itemKind: CompletionItemKind = CompletionItemKind.Variable;
this._addNameToCompletionList(name, itemKind, priorWord, completionList, this._addNameToCompletionList(
undefined, undefined, undefined, textEdit, name,
additionalTextEdits, symbol.id); itemKind,
priorWord,
completionList,
undefined,
undefined,
undefined,
textEdit,
additionalTextEdits,
symbol.id
);
} }
} }
} }
private _addNameToCompletionList(name: string, itemKind: CompletionItemKind, private _addNameToCompletionList(
filter: string, completionList: CompletionList, typeDetail?: string, name: string,
documentation?: string, autoImportText?: string, textEdit?: TextEdit, itemKind: CompletionItemKind,
additionalTextEdits?: TextEditAction[], symbolId?: number) { filter: string,
completionList: CompletionList,
typeDetail?: string,
documentation?: string,
autoImportText?: string,
textEdit?: TextEdit,
additionalTextEdits?: TextEditAction[],
symbolId?: number
) {
const similarity = StringUtils.computeCompletionSimilarity(filter, name); const similarity = StringUtils.computeCompletionSimilarity(filter, name);
if (similarity > similarityLimit) { if (similarity > similarityLimit) {
@ -1159,22 +1265,18 @@ export class CompletionProvider {
if (autoImportText) { if (autoImportText) {
// Force auto-import entries to the end. // Force auto-import entries to the end.
completionItem.sortText = completionItem.sortText = this._makeSortText(SortCategory.AutoImport, name, autoImportText);
this._makeSortText(SortCategory.AutoImport, name, autoImportText);
completionItemData.autoImportText = autoImportText; completionItemData.autoImportText = autoImportText;
} else if (SymbolNameUtils.isDunderName(name)) { } else if (SymbolNameUtils.isDunderName(name)) {
// Force dunder-named symbols to appear after all other symbols. // Force dunder-named symbols to appear after all other symbols.
completionItem.sortText = completionItem.sortText = this._makeSortText(SortCategory.DunderSymbol, name);
this._makeSortText(SortCategory.DunderSymbol, name); } else if (filter === '' && SymbolNameUtils.isPrivateOrProtectedName(name)) {
} else if (filter === '' && (SymbolNameUtils.isPrivateOrProtectedName(name))) {
// Distinguish between normal and private symbols only if there is // Distinguish between normal and private symbols only if there is
// currently no filter text. Once we get a single character to filter // currently no filter text. Once we get a single character to filter
// upon, we'll no longer differentiate. // upon, we'll no longer differentiate.
completionItem.sortText = completionItem.sortText = this._makeSortText(SortCategory.PrivateSymbol, name);
this._makeSortText(SortCategory.PrivateSymbol, name);
} else { } else {
completionItem.sortText = completionItem.sortText = this._makeSortText(SortCategory.NormalSymbol, name);
this._makeSortText(SortCategory.NormalSymbol, name);
} }
if (symbolId !== undefined) { if (symbolId !== undefined) {
@ -1228,13 +1330,11 @@ export class CompletionProvider {
private _getRecentListIndex(name: string, autoImportText: string) { private _getRecentListIndex(name: string, autoImportText: string) {
return CompletionProvider._mostRecentCompletions.findIndex( return CompletionProvider._mostRecentCompletions.findIndex(
item => item.label === name && item => item.label === name && item.autoImportText === autoImportText
item.autoImportText === autoImportText); );
} }
private _makeSortText(sortCategory: SortCategory, name: string, private _makeSortText(sortCategory: SortCategory, name: string, autoImportText = ''): string {
autoImportText = ''): string {
const recentListIndex = this._getRecentListIndex(name, autoImportText); const recentListIndex = this._getRecentListIndex(name, autoImportText);
// If the label is in the recent list, modify the category // If the label is in the recent list, modify the category
@ -1244,10 +1344,12 @@ export class CompletionProvider {
sortCategory = SortCategory.RecentAutoImport; sortCategory = SortCategory.RecentAutoImport;
} else if (sortCategory === SortCategory.ImportModuleName) { } else if (sortCategory === SortCategory.ImportModuleName) {
sortCategory = SortCategory.RecentImportModuleName; sortCategory = SortCategory.RecentImportModuleName;
} else if (sortCategory === SortCategory.Keyword || } else if (
sortCategory === SortCategory.Keyword ||
sortCategory === SortCategory.NormalSymbol || sortCategory === SortCategory.NormalSymbol ||
sortCategory === SortCategory.PrivateSymbol || sortCategory === SortCategory.PrivateSymbol ||
sortCategory === SortCategory.DunderSymbol) { sortCategory === SortCategory.DunderSymbol
) {
sortCategory = SortCategory.RecentKeywordOrSymbol; sortCategory = SortCategory.RecentKeywordOrSymbol;
} }
} }
@ -1256,9 +1358,7 @@ export class CompletionProvider {
// XX.YYYY.name // XX.YYYY.name
// where XX is the sort category // where XX is the sort category
// and YYYY is the index of the item in the MRU list // and YYYY is the index of the item in the MRU list
return this._formatInteger(sortCategory, 2) + '.' + return this._formatInteger(sortCategory, 2) + '.' + this._formatInteger(recentListIndex, 4) + '.' + name;
this._formatInteger(recentListIndex, 4) + '.' +
name;
} }
private _formatInteger(val: number, digits: number): string { private _formatInteger(val: number, digits: number): string {
@ -1292,13 +1392,12 @@ export class CompletionProvider {
return CompletionItemKind.Variable; return CompletionItemKind.Variable;
case DeclarationType.Variable: case DeclarationType.Variable:
return resolvedDeclaration.isConstant || resolvedDeclaration.isFinal ? return resolvedDeclaration.isConstant || resolvedDeclaration.isFinal
CompletionItemKind.Constant : ? CompletionItemKind.Constant
CompletionItemKind.Variable; : CompletionItemKind.Variable;
case DeclarationType.Function: case DeclarationType.Function:
return resolvedDeclaration.isMethod ? return resolvedDeclaration.isMethod ? CompletionItemKind.Method : CompletionItemKind.Function;
CompletionItemKind.Method : CompletionItemKind.Function;
case DeclarationType.Class: case DeclarationType.Class:
case DeclarationType.SpecialBuiltInClass: case DeclarationType.SpecialBuiltInClass:
@ -1318,30 +1417,35 @@ export class CompletionProvider {
importedSymbols: [] importedSymbols: []
}; };
const completions = this._importResolver.getCompletionSuggestions(this._filePath, const completions = this._importResolver.getCompletionSuggestions(
execEnvironment, moduleDescriptor, similarityLimit); this._filePath,
execEnvironment,
moduleDescriptor,
similarityLimit
);
const completionList = CompletionList.create(); const completionList = CompletionList.create();
// If we're in the middle of a "from X import Y" statement, offer // If we're in the middle of a "from X import Y" statement, offer
// the "import" keyword as a completion. // the "import" keyword as a completion.
if (!node.hasTrailingDot && node.parent && node.parent.nodeType === ParseNodeType.ImportFrom && if (
node.parent.missingImportKeyword) { !node.hasTrailingDot &&
node.parent &&
node.parent.nodeType === ParseNodeType.ImportFrom &&
node.parent.missingImportKeyword
) {
const keyword = 'import'; const keyword = 'import';
const completionItem = CompletionItem.create(keyword); const completionItem = CompletionItem.create(keyword);
completionItem.kind = CompletionItemKind.Keyword; completionItem.kind = CompletionItemKind.Keyword;
completionList.items.push(completionItem); completionList.items.push(completionItem);
completionItem.sortText = completionItem.sortText = this._makeSortText(SortCategory.Keyword, keyword);
this._makeSortText(SortCategory.Keyword, keyword);
} }
completions.forEach(completionName => { completions.forEach(completionName => {
const completionItem = CompletionItem.create(completionName); const completionItem = CompletionItem.create(completionName);
completionItem.kind = CompletionItemKind.Module; completionItem.kind = CompletionItemKind.Module;
completionList.items.push(completionItem); completionList.items.push(completionItem);
completionItem.sortText = completionItem.sortText = this._makeSortText(SortCategory.ImportModuleName, completionName);
this._makeSortText(SortCategory.ImportModuleName, completionName);
}); });
return completionList; return completionList;

8
server/src/languageService/definitionProvider.ts
View File

@ -18,9 +18,11 @@ import { ParseNodeType } from '../parser/parseNodes';
import { ParseResults } from '../parser/parser'; import { ParseResults } from '../parser/parser';
export class DefinitionProvider { export class DefinitionProvider {
static getDefinitionsForPosition(parseResults: ParseResults, position: Position, static getDefinitionsForPosition(
evaluator: TypeEvaluator): DocumentRange[] | undefined { parseResults: ParseResults,
position: Position,
evaluator: TypeEvaluator
): DocumentRange[] | undefined {
const offset = convertPositionToOffset(position, parseResults.tokenizerOutput.lines); const offset = convertPositionToOffset(position, parseResults.tokenizerOutput.lines);
if (offset === undefined) { if (offset === undefined) {
return undefined; return undefined;

69
server/src/languageService/documentSymbolProvider.ts
View File

@ -34,9 +34,13 @@ class FindSymbolTreeWalker extends ParseTreeWalker {
private _query: string | undefined; private _query: string | undefined;
private _evaluator: TypeEvaluator; private _evaluator: TypeEvaluator;
constructor(filePath: string, parseResults: ParseResults, symbolInfoResults: SymbolInformation[], constructor(
query: string | undefined, evaluator: TypeEvaluator) { filePath: string,
parseResults: ParseResults,
symbolInfoResults: SymbolInformation[],
query: string | undefined,
evaluator: TypeEvaluator
) {
super(); super();
this._filePath = filePath; this._filePath = filePath;
this._parseResults = parseResults; this._parseResults = parseResults;
@ -100,9 +104,7 @@ class FindSymbolTreeWalker extends ParseTreeWalker {
}); });
} }
private _addSymbolInformationFromDeclaration(name: string, declaration: Declaration, private _addSymbolInformationFromDeclaration(name: string, declaration: Declaration, containerName?: string) {
containerName?: string) {
if (declaration.path !== this._filePath) { if (declaration.path !== this._filePath) {
return; return;
} }
@ -179,8 +181,7 @@ function getSymbolKind(name: string, declaration: Declaration, evaluator: TypeEv
if (name === '_') { if (name === '_') {
return; return;
} }
symbolKind = declaration.isConstant || declaration.isFinal ? symbolKind = declaration.isConstant || declaration.isFinal ? SymbolKind.Constant : SymbolKind.Variable;
SymbolKind.Constant : SymbolKind.Variable;
break; break;
default: default:
@ -191,10 +192,12 @@ function getSymbolKind(name: string, declaration: Declaration, evaluator: TypeEv
return symbolKind; return symbolKind;
} }
function getDocumentSymbolsRecursive(node: AnalyzerNodeInfo.ScopedNode, function getDocumentSymbolsRecursive(
docSymbolResults: DocumentSymbol[], parseResults: ParseResults, node: AnalyzerNodeInfo.ScopedNode,
evaluator: TypeEvaluator) { docSymbolResults: DocumentSymbol[],
parseResults: ParseResults,
evaluator: TypeEvaluator
) {
const scope = AnalyzerNodeInfo.getScope(node); const scope = AnalyzerNodeInfo.getScope(node);
if (!scope) { if (!scope) {
return; return;
@ -218,10 +221,13 @@ function getDocumentSymbolsRecursive(node: AnalyzerNodeInfo.ScopedNode,
}); });
} }
function getDocumentSymbolRecursive(name: string, declaration: Declaration, function getDocumentSymbolRecursive(
evaluator: TypeEvaluator, parseResults: ParseResults, name: string,
docSymbolResults: DocumentSymbol[]) { declaration: Declaration,
evaluator: TypeEvaluator,
parseResults: ParseResults,
docSymbolResults: DocumentSymbol[]
) {
if (declaration.type === DeclarationType.Alias) { if (declaration.type === DeclarationType.Alias) {
return; return;
} }
@ -235,14 +241,14 @@ function getDocumentSymbolRecursive(name: string, declaration: Declaration,
let range = selectionRange; let range = selectionRange;
const children: DocumentSymbol[] = []; const children: DocumentSymbol[] = [];
if (declaration.type === DeclarationType.Class || if (declaration.type === DeclarationType.Class || declaration.type === DeclarationType.Function) {
declaration.type === DeclarationType.Function) {
getDocumentSymbolsRecursive(declaration.node, children, parseResults, evaluator); getDocumentSymbolsRecursive(declaration.node, children, parseResults, evaluator);
const nameRange = convertOffsetsToRange(declaration.node.start, const nameRange = convertOffsetsToRange(
declaration.node.start,
declaration.node.name.start + declaration.node.length, declaration.node.name.start + declaration.node.length,
parseResults.tokenizerOutput.lines); parseResults.tokenizerOutput.lines
);
range = nameRange; range = nameRange;
} }
@ -258,17 +264,22 @@ function getDocumentSymbolRecursive(name: string, declaration: Declaration,
} }
export class DocumentSymbolProvider { export class DocumentSymbolProvider {
static addSymbolsForDocument(symbolList: SymbolInformation[], query: string | undefined, static addSymbolsForDocument(
filePath: string, parseResults: ParseResults, evaluator: TypeEvaluator) { symbolList: SymbolInformation[],
query: string | undefined,
const symbolTreeWalker = new FindSymbolTreeWalker(filePath, parseResults, filePath: string,
symbolList, query, evaluator); parseResults: ParseResults,
evaluator: TypeEvaluator
) {
const symbolTreeWalker = new FindSymbolTreeWalker(filePath, parseResults, symbolList, query, evaluator);
symbolTreeWalker.findSymbols(); symbolTreeWalker.findSymbols();
} }
static addHierarchicalSymbolsForDocument(symbolList: DocumentSymbol[], static addHierarchicalSymbolsForDocument(
parseResults: ParseResults, evaluator: TypeEvaluator) { symbolList: DocumentSymbol[],
parseResults: ParseResults,
evaluator: TypeEvaluator
) {
getDocumentSymbolsRecursive(parseResults.parseTree, symbolList, parseResults, evaluator); getDocumentSymbolsRecursive(parseResults.parseTree, symbolList, parseResults, evaluator);
} }
} }

38
server/src/languageService/hoverProvider.ts
View File

@ -32,9 +32,11 @@ export interface HoverResults {
} }
export class HoverProvider { export class HoverProvider {
static getHoverForPosition(parseResults: ParseResults, position: Position, static getHoverForPosition(
evaluator: TypeEvaluator): HoverResults | undefined { parseResults: ParseResults,
position: Position,
evaluator: TypeEvaluator
): HoverResults | undefined {
const offset = convertPositionToOffset(position, parseResults.tokenizerOutput.lines); const offset = convertPositionToOffset(position, parseResults.tokenizerOutput.lines);
if (offset === undefined) { if (offset === undefined) {
return undefined; return undefined;
@ -84,36 +86,34 @@ export class HoverProvider {
return results.parts.length > 0 ? results : undefined; return results.parts.length > 0 ? results : undefined;
} }
private static _addResultsForDeclaration(parts: HoverTextPart[], declaration: Declaration, private static _addResultsForDeclaration(
node: NameNode, evaluator: TypeEvaluator): void { parts: HoverTextPart[],
declaration: Declaration,
node: NameNode,
evaluator: TypeEvaluator
): void {
const resolvedDecl = evaluator.resolveAliasDeclaration(declaration); const resolvedDecl = evaluator.resolveAliasDeclaration(declaration);
if (!resolvedDecl) { if (!resolvedDecl) {
this._addResultsPart(parts, `(import) ` + node.value + this._addResultsPart(parts, `(import) ` + node.value + this._getTypeText(node, evaluator), true);
this._getTypeText(node, evaluator), true);
return; return;
} }
switch (resolvedDecl.type) { switch (resolvedDecl.type) {
case DeclarationType.Intrinsic: { case DeclarationType.Intrinsic: {
this._addResultsPart(parts, node.value + this._addResultsPart(parts, node.value + this._getTypeText(node, evaluator), true);
this._getTypeText(node, evaluator), true);
this._addDocumentationPart(parts, node, evaluator); this._addDocumentationPart(parts, node, evaluator);
break; break;
} }
case DeclarationType.Variable: { case DeclarationType.Variable: {
const label = resolvedDecl.isConstant || resolvedDecl.isFinal ? const label = resolvedDecl.isConstant || resolvedDecl.isFinal ? 'constant' : 'variable';
'constant' : 'variable'; this._addResultsPart(parts, `(${label}) ` + node.value + this._getTypeText(node, evaluator), true);
this._addResultsPart(parts, `(${ label }) ` + node.value +
this._getTypeText(node, evaluator), true);
this._addDocumentationPart(parts, node, evaluator); this._addDocumentationPart(parts, node, evaluator);
break; break;
} }
case DeclarationType.Parameter: { case DeclarationType.Parameter: {
this._addResultsPart(parts, '(parameter) ' + node.value + this._addResultsPart(parts, '(parameter) ' + node.value + this._getTypeText(node, evaluator), true);
this._getTypeText(node, evaluator), true);
this._addDocumentationPart(parts, node, evaluator); this._addDocumentationPart(parts, node, evaluator);
break; break;
} }
@ -129,12 +129,10 @@ export class HoverProvider {
let label = 'function'; let label = 'function';
if (resolvedDecl.isMethod) { if (resolvedDecl.isMethod) {
const declaredType = evaluator.getTypeForDeclaration(resolvedDecl); const declaredType = evaluator.getTypeForDeclaration(resolvedDecl);
label = declaredType && isProperty(declaredType) ? label = declaredType && isProperty(declaredType) ? 'property' : 'method';
'property' : 'method';
} }
this._addResultsPart(parts, `(${ label }) ` + node.value + this._addResultsPart(parts, `(${label}) ` + node.value + this._getTypeText(node, evaluator), true);
this._getTypeText(node, evaluator), true);
this._addDocumentationPart(parts, node, evaluator); this._addDocumentationPart(parts, node, evaluator);
break; break;
} }

65
server/src/languageService/importSorter.ts
View File

@ -33,11 +33,11 @@ export class ImportSorter {
sort(): TextEditAction[] { sort(): TextEditAction[] {
const actions: TextEditAction[] = []; const actions: TextEditAction[] = [];
const importStatements = ImportStatementUtils.getTopLevelImports( const importStatements = ImportStatementUtils.getTopLevelImports(this._parseResults.parseTree);
this._parseResults.parseTree);
const sortedStatements = importStatements.orderedImports. const sortedStatements = importStatements.orderedImports
map(s => s).sort((a, b) => { .map(s => s)
.sort((a, b) => {
return this._compareImportStatements(a, b); return this._compareImportStatements(a, b);
}); });
@ -46,23 +46,19 @@ export class ImportSorter {
return []; return [];
} }
const primaryRange = this._getPrimaryReplacementRange( const primaryRange = this._getPrimaryReplacementRange(importStatements.orderedImports);
importStatements.orderedImports);
actions.push({ actions.push({
range: primaryRange, range: primaryRange,
replacementText: this._generateSortedImportText(sortedStatements) replacementText: this._generateSortedImportText(sortedStatements)
}); });
this._addSecondaryReplacementRanges( this._addSecondaryReplacementRanges(importStatements.orderedImports, actions);
importStatements.orderedImports, actions);
return actions; return actions;
} }
private _compareImportStatements(a: ImportStatementUtils.ImportStatement, private _compareImportStatements(a: ImportStatementUtils.ImportStatement, b: ImportStatementUtils.ImportStatement) {
b: ImportStatementUtils.ImportStatement) {
const aImportGroup = this._getImportGroup(a); const aImportGroup = this._getImportGroup(a);
const bImportGroup = this._getImportGroup(b); const bImportGroup = this._getImportGroup(b);
@ -72,16 +68,17 @@ export class ImportSorter {
return 1; return 1;
} }
return (a.moduleName < b.moduleName) ? -1 : 1; return a.moduleName < b.moduleName ? -1 : 1;
} }
private _getImportGroup(statement: ImportStatementUtils.ImportStatement): ImportGroup { private _getImportGroup(statement: ImportStatementUtils.ImportStatement): ImportGroup {
if (statement.importResult) { if (statement.importResult) {
if (statement.importResult.importType === ImportType.BuiltIn) { if (statement.importResult.importType === ImportType.BuiltIn) {
return ImportGroup.BuiltIn; return ImportGroup.BuiltIn;
} else if (statement.importResult.importType === ImportType.ThirdParty || } else if (
statement.importResult.isLocalTypingsFile) { statement.importResult.importType === ImportType.ThirdParty ||
statement.importResult.isLocalTypingsFile
) {
return ImportGroup.ThirdParty; return ImportGroup.ThirdParty;
} }
@ -98,9 +95,7 @@ export class ImportSorter {
// Determines the text range for the existing primary block of import statements. // Determines the text range for the existing primary block of import statements.
// If there are other blocks of import statements separated by other statements, // If there are other blocks of import statements separated by other statements,
// we'll ignore these other blocks for now. // we'll ignore these other blocks for now.
private _getPrimaryReplacementRange(statements: ImportStatementUtils.ImportStatement[]): private _getPrimaryReplacementRange(statements: ImportStatementUtils.ImportStatement[]): Range {
Range {
let statementLimit = statements.findIndex(s => s.followsNonImportStatement); let statementLimit = statements.findIndex(s => s.followsNonImportStatement);
if (statementLimit < 0) { if (statementLimit < 0) {
statementLimit = statements.length; statementLimit = statements.length;
@ -108,18 +103,17 @@ export class ImportSorter {
const lastStatement = statements[statementLimit - 1].node; const lastStatement = statements[statementLimit - 1].node;
return { return {
start: convertOffsetToPosition( start: convertOffsetToPosition(statements[0].node.start, this._parseResults.tokenizerOutput.lines),
statements[0].node.start, this._parseResults.tokenizerOutput.lines), end: convertOffsetToPosition(TextRange.getEnd(lastStatement), this._parseResults.tokenizerOutput.lines)
end: convertOffsetToPosition(
TextRange.getEnd(lastStatement), this._parseResults.tokenizerOutput.lines)
}; };
} }
// If import statements are separated by other statements, we will remove the old // If import statements are separated by other statements, we will remove the old
// secondary blocks. // secondary blocks.
private _addSecondaryReplacementRanges(statements: ImportStatementUtils.ImportStatement[], private _addSecondaryReplacementRanges(
actions: TextEditAction[]) { statements: ImportStatementUtils.ImportStatement[],
actions: TextEditAction[]
) {
let secondaryBlockStart = statements.findIndex(s => s.followsNonImportStatement); let secondaryBlockStart = statements.findIndex(s => s.followsNonImportStatement);
if (secondaryBlockStart < 0) { if (secondaryBlockStart < 0) {
return; return;
@ -127,7 +121,8 @@ export class ImportSorter {
while (true) { while (true) {
let secondaryBlockLimit = statements.findIndex( let secondaryBlockLimit = statements.findIndex(
(s, index) => index > secondaryBlockStart && s.followsNonImportStatement); (s, index) => index > secondaryBlockStart && s.followsNonImportStatement
);
if (secondaryBlockLimit < 0) { if (secondaryBlockLimit < 0) {
secondaryBlockLimit = statements.length; secondaryBlockLimit = statements.length;
} }
@ -136,10 +131,12 @@ export class ImportSorter {
range: { range: {
start: convertOffsetToPosition( start: convertOffsetToPosition(
statements[secondaryBlockStart].node.start, statements[secondaryBlockStart].node.start,
this._parseResults.tokenizerOutput.lines), this._parseResults.tokenizerOutput.lines
),
end: convertOffsetToPosition( end: convertOffsetToPosition(
TextRange.getEnd(statements[secondaryBlockLimit - 1].node), TextRange.getEnd(statements[secondaryBlockLimit - 1].node),
this._parseResults.tokenizerOutput.lines) this._parseResults.tokenizerOutput.lines
)
}, },
replacementText: '' replacementText: ''
}); });
@ -165,11 +162,9 @@ export class ImportSorter {
let importLine: string; let importLine: string;
if (statement.node.nodeType === ParseNodeType.Import) { if (statement.node.nodeType === ParseNodeType.Import) {
importLine = this._formatImportNode(statement.subnode!, importLine = this._formatImportNode(statement.subnode!, statement.moduleName);
statement.moduleName);
} else { } else {
importLine = this._formatImportFromNode(statement.node, importLine = this._formatImportFromNode(statement.node, statement.moduleName);
statement.moduleName);
} }
// If this isn't the last statement, add a newline. // If this isn't the last statement, add a newline.
@ -193,9 +188,9 @@ export class ImportSorter {
} }
private _formatImportFromNode(node: ImportFromNode, moduleName: string): string { private _formatImportFromNode(node: ImportFromNode, moduleName: string): string {
const symbols = node.imports. const symbols = node.imports
sort((a, b) => this._compareSymbols(a, b)). .sort((a, b) => this._compareSymbols(a, b))
map(symbol => { .map(symbol => {
let symbolText = symbol.name.value; let symbolText = symbol.name.value;
if (symbol.alias) { if (symbol.alias) {
symbolText += ` as ${symbol.alias.value}`; symbolText += ` as ${symbol.alias.value}`;

32
server/src/languageService/quickActions.ts
View File

@ -18,9 +18,7 @@ import { ParseNode, ParseNodeType } from '../parser/parseNodes';
import { ParseResults } from '../parser/parser'; import { ParseResults } from '../parser/parser';
import { ImportSorter } from './importSorter'; import { ImportSorter } from './importSorter';
export function performQuickAction(command: string, args: any[], export function performQuickAction(command: string, args: any[], parseResults: ParseResults) {
parseResults: ParseResults) {
if (command === Commands.orderImports) { if (command === Commands.orderImports) {
const importSorter = new ImportSorter(parseResults); const importSorter = new ImportSorter(parseResults);
return importSorter.sort(); return importSorter.sort();
@ -36,9 +34,7 @@ export function performQuickAction(command: string, args: any[],
// Inserts text into the document to wrap an existing type annotation // Inserts text into the document to wrap an existing type annotation
// with "Optional[X]". // with "Optional[X]".
function _addMissingOptionalToParam(parseResults: ParseResults, function _addMissingOptionalToParam(parseResults: ParseResults, offset: number): TextEditAction[] {
offset: number): TextEditAction[] {
let node: ParseNode | undefined = ParseTreeUtils.findNodeByOffset(parseResults.parseTree, offset); let node: ParseNode | undefined = ParseTreeUtils.findNodeByOffset(parseResults.parseTree, offset);
while (node) { while (node) {
if (node.nodeType === ParseNodeType.Parameter) { if (node.nodeType === ParseNodeType.Parameter) {
@ -54,10 +50,8 @@ function _addMissingOptionalToParam(parseResults: ParseResults,
const editActions: TextEditAction[] = []; const editActions: TextEditAction[] = [];
const startPos = convertOffsetToPosition( const startPos = convertOffsetToPosition(node.typeAnnotation.start, parseResults.tokenizerOutput.lines);
node.typeAnnotation.start, parseResults.tokenizerOutput.lines); const endPos = convertOffsetToPosition(TextRange.getEnd(node.typeAnnotation), parseResults.tokenizerOutput.lines);
const endPos = convertOffsetToPosition(
TextRange.getEnd(node.typeAnnotation), parseResults.tokenizerOutput.lines);
editActions.push({ editActions.push({
range: { start: startPos, end: startPos }, range: { start: startPos, end: startPos },
@ -69,19 +63,25 @@ function _addMissingOptionalToParam(parseResults: ParseResults,
}); });
// Add the import statement if necessary. // Add the import statement if necessary.
const importStatements = ImportStatementUtils.getTopLevelImports( const importStatements = ImportStatementUtils.getTopLevelImports(parseResults.parseTree);
parseResults.parseTree); const importStatement = importStatements.orderedImports.find(imp => imp.moduleName === 'typing');
const importStatement = importStatements.orderedImports.find(
imp => imp.moduleName === 'typing');
// If there's an existing import statement, insert into it. // If there's an existing import statement, insert into it.
if (importStatement && importStatement.node.nodeType === ParseNodeType.ImportFrom) { if (importStatement && importStatement.node.nodeType === ParseNodeType.ImportFrom) {
const additionalEditActions = ImportStatementUtils.getTextEditsForAutoImportSymbolAddition( const additionalEditActions = ImportStatementUtils.getTextEditsForAutoImportSymbolAddition(
'Optional', importStatement, parseResults); 'Optional',
importStatement,
parseResults
);
editActions.push(...additionalEditActions); editActions.push(...additionalEditActions);
} else { } else {
const additionalEditActions = ImportStatementUtils.getTextEditsForAutoImportInsertion( const additionalEditActions = ImportStatementUtils.getTextEditsForAutoImportInsertion(
'Optional', importStatements, 'typing', ImportType.BuiltIn, parseResults); 'Optional',
importStatements,
'typing',
ImportType.BuiltIn,
parseResults
);
editActions.push(...additionalEditActions); editActions.push(...additionalEditActions);
} }

62
server/src/languageService/referencesProvider.ts
View File

@ -34,10 +34,13 @@ class FindReferencesTreeWalker extends ParseTreeWalker {
private _includeDeclaration: boolean; private _includeDeclaration: boolean;
private _evaluator: TypeEvaluator; private _evaluator: TypeEvaluator;
constructor(parseResults: ParseResults, filePath: string, constructor(
referencesResult: ReferencesResult, includeDeclaration: boolean, parseResults: ParseResults,
evaluator: TypeEvaluator) { filePath: string,
referencesResult: ReferencesResult,
includeDeclaration: boolean,
evaluator: TypeEvaluator
) {
super(); super();
this._parseResults = parseResults; this._parseResults = parseResults;
this._filePath = filePath; this._filePath = filePath;
@ -68,7 +71,10 @@ class FindReferencesTreeWalker extends ParseTreeWalker {
path: this._filePath, path: this._filePath,
range: { range: {
start: convertOffsetToPosition(node.start, this._parseResults.tokenizerOutput.lines), start: convertOffsetToPosition(node.start, this._parseResults.tokenizerOutput.lines),
end: convertOffsetToPosition(TextRange.getEnd(node), this._parseResults.tokenizerOutput.lines) end: convertOffsetToPosition(
TextRange.getEnd(node),
this._parseResults.tokenizerOutput.lines
)
} }
}); });
} }
@ -87,15 +93,19 @@ class FindReferencesTreeWalker extends ParseTreeWalker {
// The reference results declarations are already resolved, so we don't // The reference results declarations are already resolved, so we don't
// need to call resolveAliasDeclaration on them. // need to call resolveAliasDeclaration on them.
return this._referencesResult.declarations.some(decl => return this._referencesResult.declarations.some(decl =>
DeclarationUtils.areDeclarationsSame(decl, resolvedDecl)); DeclarationUtils.areDeclarationsSame(decl, resolvedDecl)
);
} }
} }
export class ReferencesProvider { export class ReferencesProvider {
static getReferencesForPosition(parseResults: ParseResults, filePath: string, static getReferencesForPosition(
position: Position, includeDeclaration: boolean, parseResults: ParseResults,
evaluator: TypeEvaluator): ReferencesResult | undefined { filePath: string,
position: Position,
includeDeclaration: boolean,
evaluator: TypeEvaluator
): ReferencesResult | undefined {
const offset = convertPositionToOffset(position, parseResults.tokenizerOutput.lines); const offset = convertPositionToOffset(position, parseResults.tokenizerOutput.lines);
if (offset === undefined) { if (offset === undefined) {
return undefined; return undefined;
@ -133,7 +143,8 @@ export class ReferencesProvider {
// Parameters are local to a scope, so they don't require a global search. // Parameters are local to a scope, so they don't require a global search.
// If it's a named argument referring to a parameter, we still need to perform // If it's a named argument referring to a parameter, we still need to perform
// the global search. // the global search.
const requiresGlobalSearch = symbolDeclType !== DeclarationType.Parameter || const requiresGlobalSearch =
symbolDeclType !== DeclarationType.Parameter ||
(node.parent !== undefined && node.parent.nodeType === ParseNodeType.Argument); (node.parent !== undefined && node.parent.nodeType === ParseNodeType.Argument);
const results: ReferencesResult = { const results: ReferencesResult = {
@ -143,19 +154,32 @@ export class ReferencesProvider {
locations: [] locations: []
}; };
const refTreeWalker = new FindReferencesTreeWalker(parseResults, const refTreeWalker = new FindReferencesTreeWalker(
filePath, results, includeDeclaration, evaluator); parseResults,
filePath,
results,
includeDeclaration,
evaluator
);
refTreeWalker.findReferences(); refTreeWalker.findReferences();
return results; return results;
} }
static addReferences(parseResults: ParseResults, filePath: string, static addReferences(
referencesResult: ReferencesResult, includeDeclaration: boolean, parseResults: ParseResults,
evaluator: TypeEvaluator): void { filePath: string,
referencesResult: ReferencesResult,
const refTreeWalker = new FindReferencesTreeWalker(parseResults, includeDeclaration: boolean,
filePath, referencesResult, includeDeclaration, evaluator); evaluator: TypeEvaluator
): void {
const refTreeWalker = new FindReferencesTreeWalker(
parseResults,
filePath,
referencesResult,
includeDeclaration,
evaluator
);
refTreeWalker.findReferences(); refTreeWalker.findReferences();
} }
} }

9
server/src/languageService/signatureHelpProvider.ts
View File

@ -36,10 +36,11 @@ export interface SignatureHelpResults {
} }
export class SignatureHelpProvider { export class SignatureHelpProvider {
static getSignatureHelpForPosition(parseResults: ParseResults, position: Position, static getSignatureHelpForPosition(
evaluator: TypeEvaluator): parseResults: ParseResults,
SignatureHelpResults | undefined { position: Position,
evaluator: TypeEvaluator
): SignatureHelpResults | undefined {
const offset = convertPositionToOffset(position, parseResults.tokenizerOutput.lines); const offset = convertPositionToOffset(position, parseResults.tokenizerOutput.lines);
if (offset === undefined) { if (offset === undefined) {
return undefined; return undefined;

78
server/src/parser/characters.ts
View File

@ -51,12 +51,16 @@ export function isIdentifierStartChar(ch: number) {
export function isIdentifierChar(ch: number) { export function isIdentifierChar(ch: number) {
if (ch < _identifierCharFastTableSize) { if (ch < _identifierCharFastTableSize) {
return _identifierCharFastTable[ch] === CharCategory.StartIdentifierChar || return (
_identifierCharFastTable[ch] === CharCategory.IdentifierChar; _identifierCharFastTable[ch] === CharCategory.StartIdentifierChar ||
_identifierCharFastTable[ch] === CharCategory.IdentifierChar
);
} }
return _identifierCharMap[ch] === CharCategory.StartIdentifierChar || return (
_identifierCharMap[ch] === CharCategory.IdentifierChar; _identifierCharMap[ch] === CharCategory.StartIdentifierChar ||
_identifierCharMap[ch] === CharCategory.IdentifierChar
);
} }
export function isWhiteSpace(ch: number): boolean { export function isWhiteSpace(ch: number): boolean {
@ -68,11 +72,11 @@ export function isLineBreak(ch: number): boolean {
} }
export function isNumber(ch: number): boolean { export function isNumber(ch: number): boolean {
return ch >= Char._0 && ch <= Char._9 || ch === Char.Underscore; return (ch >= Char._0 && ch <= Char._9) || ch === Char.Underscore;
} }
export function isDecimal(ch: number): boolean { export function isDecimal(ch: number): boolean {
return ch >= Char._0 && ch <= Char._9 || ch === Char.Underscore; return (ch >= Char._0 && ch <= Char._9) || ch === Char.Underscore;
} }
export function isHex(ch: number): boolean { export function isHex(ch: number): boolean {
@ -80,7 +84,7 @@ export function isHex(ch: number): boolean {
} }
export function isOctal(ch: number): boolean { export function isOctal(ch: number): boolean {
return ch >= Char._0 && ch <= Char._7 || ch === Char.Underscore; return (ch >= Char._0 && ch <= Char._7) || ch === Char.Underscore;
} }
export function isBinary(ch: number): boolean { export function isBinary(ch: number): boolean {
@ -89,21 +93,55 @@ export function isBinary(ch: number): boolean {
// Underscore is explicitly allowed to start an identifier. // Underscore is explicitly allowed to start an identifier.
// Characters with the Other_ID_Start property. // Characters with the Other_ID_Start property.
const _specialStartIdentifierChars: unicode.UnicodeRangeTable = [Char.Underscore, const _specialStartIdentifierChars: unicode.UnicodeRangeTable = [
0x1885, 0x1886, 0x2118, 0x212E, 0x309B, 0x309C]; Char.Underscore,
0x1885,
0x1886,
0x2118,
0x212e,
0x309b,
0x309c
];
const _startIdentifierCharRanges = [_specialStartIdentifierChars, unicode.unicodeLu, const _startIdentifierCharRanges = [
unicode.unicodeLl, unicode.unicodeLt, unicode.unicodeLo, unicode.unicodeLm, unicode.unicodeNl]; _specialStartIdentifierChars,
unicode.unicodeLu,
unicode.unicodeLl,
unicode.unicodeLt,
unicode.unicodeLo,
unicode.unicodeLm,
unicode.unicodeNl
];
// Characters with the Other_ID_Start property. // Characters with the Other_ID_Start property.
const _specialIdentifierChars: unicode.UnicodeRangeTable = [0x00B7, 0x0387, 0x1369, const _specialIdentifierChars: unicode.UnicodeRangeTable = [
0x136A, 0x136B, 0x136C, 0x136D, 0x136E, 0x136F, 0x1370, 0x1371, 0x19DA]; 0x00b7,
0x0387,
0x1369,
0x136a,
0x136b,
0x136c,
0x136d,
0x136e,
0x136f,
0x1370,
0x1371,
0x19da
];
const _identifierCharRanges = [_specialIdentifierChars, unicode.unicodeMn, unicode.unicodeMc, const _identifierCharRanges = [
unicode.unicodeNd, unicode.unicodePc]; _specialIdentifierChars,
unicode.unicodeMn,
unicode.unicodeMc,
unicode.unicodeNd,
unicode.unicodePc
];
function _buildIdentifierLookupTableFromUnicodeRangeTable(table: unicode.UnicodeRangeTable, function _buildIdentifierLookupTableFromUnicodeRangeTable(
category: CharCategory, fastTableOnly: boolean) { table: unicode.UnicodeRangeTable,
category: CharCategory,
fastTableOnly: boolean
) {
for (let entryIndex = 0; entryIndex < table.length; entryIndex++) { for (let entryIndex = 0; entryIndex < table.length; entryIndex++) {
const entry = table[entryIndex]; const entry = table[entryIndex];
let rangeStart: number; let rangeStart: number;
@ -135,13 +173,11 @@ function _buildIdentifierLookupTable(fastTableOnly: boolean) {
_identifierCharFastTable.fill(CharCategory.NotIdentifierChar); _identifierCharFastTable.fill(CharCategory.NotIdentifierChar);
_identifierCharRanges.forEach(table => { _identifierCharRanges.forEach(table => {
_buildIdentifierLookupTableFromUnicodeRangeTable( _buildIdentifierLookupTableFromUnicodeRangeTable(table, CharCategory.IdentifierChar, fastTableOnly);
table, CharCategory.IdentifierChar, fastTableOnly);
}); });
_startIdentifierCharRanges.forEach(table => { _startIdentifierCharRanges.forEach(table => {
_buildIdentifierLookupTableFromUnicodeRangeTable( _buildIdentifierLookupTableFromUnicodeRangeTable(table, CharCategory.StartIdentifierChar, fastTableOnly);
table, CharCategory.StartIdentifierChar, fastTableOnly);
}); });
} }

201
server/src/parser/parseNodes.ts
View File

@ -9,8 +9,16 @@
*/ */
import { TextRange } from '../common/textRange'; import { TextRange } from '../common/textRange';
import { IdentifierToken, KeywordToken, KeywordType, import {
NumberToken, OperatorType, StringToken, Token, TokenType } from './tokenizerTypes'; IdentifierToken,
KeywordToken,
KeywordType,
NumberToken,
OperatorType,
StringToken,
Token,
TokenType
} from './tokenizerTypes';
export const enum ParseNodeType { export const enum ParseNodeType {
Error, // 0 Error, // 0
@ -172,9 +180,12 @@ export interface IfNode extends ParseNodeBase {
} }
export namespace IfNode { export namespace IfNode {
export function create(ifOrElifToken: Token, testExpression: ExpressionNode, export function create(
ifSuite: SuiteNode, elseSuite?: SuiteNode) { ifOrElifToken: Token,
testExpression: ExpressionNode,
ifSuite: SuiteNode,
elseSuite?: SuiteNode
) {
const node: IfNode = { const node: IfNode = {
start: ifOrElifToken.start, start: ifOrElifToken.start,
length: ifOrElifToken.length, length: ifOrElifToken.length,
@ -236,9 +247,12 @@ export interface ForNode extends ParseNodeBase {
} }
export namespace ForNode { export namespace ForNode {
export function create(forToken: Token, targetExpression: ExpressionNode, export function create(
iterableExpression: ExpressionNode, forSuite: SuiteNode) { forToken: Token,
targetExpression: ExpressionNode,
iterableExpression: ExpressionNode,
forSuite: SuiteNode
) {
const node: ForNode = { const node: ForNode = {
start: forToken.start, start: forToken.start,
length: forToken.length, length: forToken.length,
@ -588,18 +602,57 @@ export namespace StatementListNode {
} }
} }
export type StatementNode = IfNode | WhileNode | ForNode | TryNode | export type StatementNode =
FunctionNode | ClassNode | WithNode | StatementListNode | ErrorNode; | IfNode
| WhileNode
| ForNode
| TryNode
| FunctionNode
| ClassNode
| WithNode
| StatementListNode
| ErrorNode;
export type SmallStatementNode = ExpressionNode | DelNode | PassNode | export type SmallStatementNode =
ImportNode | GlobalNode | NonlocalNode | AssertNode; | ExpressionNode
| DelNode
| PassNode
| ImportNode
| GlobalNode
| NonlocalNode
| AssertNode;
export type ExpressionNode = ErrorNode | UnaryOperationNode | BinaryOperationNode | export type ExpressionNode =
AssignmentNode | TypeAnnotationNode | AssignmentExpressionNode | AugmentedAssignmentNode | | ErrorNode
AwaitNode | TernaryNode | UnpackNode | TupleNode | CallNode | ListComprehensionNode | IndexNode | | UnaryOperationNode
SliceNode | YieldNode | YieldFromNode | MemberAccessNode | LambdaNode | NameNode | ConstantNode | | BinaryOperationNode
EllipsisNode | NumberNode | StringNode | FormatStringNode | StringListNode | DictionaryNode | | AssignmentNode
DictionaryExpandEntryNode | ListNode | SetNode; | TypeAnnotationNode
| AssignmentExpressionNode
| AugmentedAssignmentNode
| AwaitNode
| TernaryNode
| UnpackNode
| TupleNode
| CallNode
| ListComprehensionNode
| IndexNode
| SliceNode
| YieldNode
| YieldFromNode
| MemberAccessNode
| LambdaNode
| NameNode
| ConstantNode
| EllipsisNode
| NumberNode
| StringNode
| FormatStringNode
| StringListNode
| DictionaryNode
| DictionaryExpandEntryNode
| ListNode
| SetNode;
export function isExpressionNode(node: ParseNode) { export function isExpressionNode(node: ParseNode) {
switch (node.nodeType) { switch (node.nodeType) {
@ -645,9 +698,7 @@ export interface ErrorNode extends ParseNodeBase {
} }
export namespace ErrorNode { export namespace ErrorNode {
export function create(initialRange: TextRange, category: ErrorExpressionCategory, export function create(initialRange: TextRange, category: ErrorExpressionCategory, child?: ExpressionNode) {
child?: ExpressionNode) {
const node: ErrorNode = { const node: ErrorNode = {
start: initialRange.start, start: initialRange.start,
length: initialRange.length, length: initialRange.length,
@ -699,9 +750,7 @@ export interface BinaryOperationNode extends ParseNodeBase {
} }
export namespace BinaryOperationNode { export namespace BinaryOperationNode {
export function create(leftExpression: ExpressionNode, rightExpression: ExpressionNode, export function create(leftExpression: ExpressionNode, rightExpression: ExpressionNode, operator: OperatorType) {
operator: OperatorType) {
const node: BinaryOperationNode = { const node: BinaryOperationNode = {
start: leftExpression.start, start: leftExpression.start,
length: leftExpression.length, length: leftExpression.length,
@ -813,9 +862,12 @@ export interface AugmentedAssignmentNode extends ParseNodeBase {
} }
export namespace AugmentedAssignmentNode { export namespace AugmentedAssignmentNode {
export function create(leftExpression: ExpressionNode, rightExpression: ExpressionNode, export function create(
operator: OperatorType, destExpression: ExpressionNode) { leftExpression: ExpressionNode,
rightExpression: ExpressionNode,
operator: OperatorType,
destExpression: ExpressionNode
) {
const node: AugmentedAssignmentNode = { const node: AugmentedAssignmentNode = {
start: leftExpression.start, start: leftExpression.start,
length: leftExpression.length, length: leftExpression.length,
@ -868,9 +920,11 @@ export interface TernaryNode extends ParseNodeBase {
} }
export namespace TernaryNode { export namespace TernaryNode {
export function create(ifExpression: ExpressionNode, testExpression: ExpressionNode, export function create(
elseExpression: ExpressionNode) { ifExpression: ExpressionNode,
testExpression: ExpressionNode,
elseExpression: ExpressionNode
) {
const node: TernaryNode = { const node: TernaryNode = {
start: ifExpression.start, start: ifExpression.start,
length: ifExpression.length, length: ifExpression.length,
@ -1260,9 +1314,12 @@ export interface FormatStringNode extends ParseNodeBase {
} }
export namespace FormatStringNode { export namespace FormatStringNode {
export function create(token: StringToken, unescapedValue: string, hasUnescapeErrors: boolean, export function create(
expressions: ExpressionNode[]) { token: StringToken,
unescapedValue: string,
hasUnescapeErrors: boolean,
expressions: ExpressionNode[]
) {
const node: FormatStringNode = { const node: FormatStringNode = {
start: token.start, start: token.start,
length: token.length, length: token.length,
@ -1738,19 +1795,71 @@ export namespace RaiseNode {
} }
} }
export type ParseNode = ErrorNode | ArgumentNode | AssertNode | AssignmentExpressionNode | export type ParseNode =
AssignmentNode | AugmentedAssignmentNode | AwaitNode | BinaryOperationNode | | ErrorNode
BreakNode | CallNode | ClassNode | ConstantNode | ContinueNode | | ArgumentNode
DecoratorNode | DelNode | DictionaryNode | DictionaryEntryNode | DictionaryExpandEntryNode | | AssertNode
DictionaryKeyEntryNode | EllipsisNode | IfNode | ImportNode | ImportAsNode | ImportFromNode | | AssignmentExpressionNode
ImportFromAsNode | IndexNode | IndexItemsNode | ExceptNode | ForNode | FormatStringNode | | AssignmentNode
FunctionNode | GlobalNode | LambdaNode | ListNode | ListComprehensionNode | ListComprehensionForNode | | AugmentedAssignmentNode
ListComprehensionIfNode | MemberAccessNode | ModuleNameNode | ModuleNode | NameNode | | AwaitNode
NonlocalNode | NumberNode | ParameterNode | PassNode | RaiseNode | ReturnNode | SetNode | | BinaryOperationNode
SliceNode | StatementListNode | StringListNode | StringNode | SuiteNode | | BreakNode
TernaryNode | TupleNode | TryNode | TypeAnnotationNode | | CallNode
UnaryOperationNode | UnpackNode | WhileNode | WithNode | WithItemNode | | ClassNode
YieldNode | YieldFromNode; | ConstantNode
| ContinueNode
| DecoratorNode
| DelNode
| DictionaryNode
| DictionaryEntryNode
| DictionaryExpandEntryNode
| DictionaryKeyEntryNode
| EllipsisNode
| IfNode
| ImportNode
| ImportAsNode
| ImportFromNode
| ImportFromAsNode
| IndexNode
| IndexItemsNode
| ExceptNode
| ForNode
| FormatStringNode
| FunctionNode
| GlobalNode
| LambdaNode
| ListNode
| ListComprehensionNode
| ListComprehensionForNode
| ListComprehensionIfNode
| MemberAccessNode
| ModuleNameNode
| ModuleNode
| NameNode
| NonlocalNode
| NumberNode
| ParameterNode
| PassNode
| RaiseNode
| ReturnNode
| SetNode
| SliceNode
| StatementListNode
| StringListNode
| StringNode
| SuiteNode
| TernaryNode
| TupleNode
| TryNode
| TypeAnnotationNode
| UnaryOperationNode
| UnpackNode
| WhileNode
| WithNode
| WithItemNode
| YieldNode
| YieldFromNode;
export type EvaluationScopeNode = LambdaNode | FunctionNode | ModuleNode | ClassNode | ListComprehensionNode; export type EvaluationScopeNode = LambdaNode | FunctionNode | ModuleNode | ClassNode | ListComprehensionNode;
export type ExecutionScopeNode = LambdaNode | FunctionNode | ModuleNode; export type ExecutionScopeNode = LambdaNode | FunctionNode | ModuleNode;

495
server/src/parser/parser.ts
View File

@ -19,24 +19,97 @@ import { latestStablePythonVersion, PythonVersion } from '../common/pythonVersio
import { TextRange } from '../common/textRange'; import { TextRange } from '../common/textRange';
import { TextRangeCollection } from '../common/textRangeCollection'; import { TextRangeCollection } from '../common/textRangeCollection';
import { timingStats } from '../common/timing'; import { timingStats } from '../common/timing';
import { ArgumentCategory, ArgumentNode, AssertNode, AssignmentExpressionNode, AssignmentNode, import {
AugmentedAssignmentNode, AwaitNode, BinaryOperationNode, BreakNode, CallNode, ArgumentCategory,
ClassNode, ConstantNode, ContinueNode, DecoratorNode, DelNode, DictionaryEntryNode, ArgumentNode,
DictionaryExpandEntryNode, DictionaryKeyEntryNode, DictionaryNode, EllipsisNode, AssertNode,
ErrorExpressionCategory, ErrorNode, ExceptNode, ExpressionNode, extendRange, AssignmentExpressionNode,
FormatStringNode, ForNode, FunctionNode, getNextNodeId, GlobalNode, IfNode, ImportAsNode, AssignmentNode,
ImportFromAsNode, ImportFromNode, ImportNode, IndexItemsNode, IndexNode, LambdaNode, AugmentedAssignmentNode,
ListComprehensionForNode, ListComprehensionIfNode, ListComprehensionIterNode, AwaitNode,
ListComprehensionNode, ListNode, MemberAccessNode, ModuleNameNode, BinaryOperationNode,
ModuleNode, NameNode, NonlocalNode, NumberNode, ParameterCategory, ParameterNode, BreakNode,
ParseNode, ParseNodeType, PassNode, RaiseNode, ReturnNode, SetNode, SliceNode, CallNode,
StatementListNode, StatementNode, StringListNode, StringNode, SuiteNode, TernaryNode, ClassNode,
TryNode, TupleNode, TypeAnnotationNode, UnaryOperationNode, UnpackNode, ConstantNode,
WhileNode, WithItemNode, WithNode, YieldFromNode, YieldNode } from './parseNodes'; ContinueNode,
DecoratorNode,
DelNode,
DictionaryEntryNode,
DictionaryExpandEntryNode,
DictionaryKeyEntryNode,
DictionaryNode,
EllipsisNode,
ErrorExpressionCategory,
ErrorNode,
ExceptNode,
ExpressionNode,
extendRange,
FormatStringNode,
ForNode,
FunctionNode,
getNextNodeId,
GlobalNode,
IfNode,
ImportAsNode,
ImportFromAsNode,
ImportFromNode,
ImportNode,
IndexItemsNode,
IndexNode,
LambdaNode,
ListComprehensionForNode,
ListComprehensionIfNode,
ListComprehensionIterNode,
ListComprehensionNode,
ListNode,
MemberAccessNode,
ModuleNameNode,
ModuleNode,
NameNode,
NonlocalNode,
NumberNode,
ParameterCategory,
ParameterNode,
ParseNode,
ParseNodeType,
PassNode,
RaiseNode,
ReturnNode,
SetNode,
SliceNode,
StatementListNode,
StatementNode,
StringListNode,
StringNode,
SuiteNode,
TernaryNode,
TryNode,
TupleNode,
TypeAnnotationNode,
UnaryOperationNode,
UnpackNode,
WhileNode,
WithItemNode,
WithNode,
YieldFromNode,
YieldNode
} from './parseNodes';
import * as StringTokenUtils from './stringTokenUtils'; import * as StringTokenUtils from './stringTokenUtils';
import { Tokenizer, TokenizerOutput } from './tokenizer'; import { Tokenizer, TokenizerOutput } from './tokenizer';
import { DedentToken, IdentifierToken, KeywordToken, KeywordType, NumberToken, OperatorToken, import {
OperatorType, StringToken, StringTokenFlags, Token, TokenType } from './tokenizerTypes'; DedentToken,
IdentifierToken,
KeywordToken,
KeywordType,
NumberToken,
OperatorToken,
OperatorType,
StringToken,
StringTokenFlags,
Token,
TokenType
} from './tokenizerTypes';
interface ExpressionListResult { interface ExpressionListResult {
list: ExpressionNode[]; list: ExpressionNode[];
@ -95,12 +168,9 @@ export class Parser {
private _containsWildcardImport = false; private _containsWildcardImport = false;
private _assignmentExpressionsAllowed = true; private _assignmentExpressionsAllowed = true;
parseSourceFile(fileContents: string, parseOptions: ParseOptions, parseSourceFile(fileContents: string, parseOptions: ParseOptions, diagSink: DiagnosticSink): ParseResults {
diagSink: DiagnosticSink): ParseResults {
timingStats.tokenizeFileTime.timeOperation(() => { timingStats.tokenizeFileTime.timeOperation(() => {
this._startNewParse(fileContents, 0, fileContents.length, this._startNewParse(fileContents, 0, fileContents.length, parseOptions, diagSink);
parseOptions, diagSink);
}); });
const moduleNode = ModuleNode.create({ start: 0, length: fileContents.length }); const moduleNode = ModuleNode.create({ start: 0, length: fileContents.length });
@ -139,9 +209,13 @@ export class Parser {
}; };
} }
parseTextExpression(fileContents: string, textOffset: number, textLength: number, parseTextExpression(
parseOptions: ParseOptions, parseTypeAnnotation: boolean): ParseExpressionTextResults { fileContents: string,
textOffset: number,
textLength: number,
parseOptions: ParseOptions,
parseTypeAnnotation: boolean
): ParseExpressionTextResults {
const diagSink = new DiagnosticSink(); const diagSink = new DiagnosticSink();
this._startNewParse(fileContents, textOffset, textLength, parseOptions, diagSink); this._startNewParse(fileContents, textOffset, textLength, parseOptions, diagSink);
@ -169,9 +243,13 @@ export class Parser {
}; };
} }
private _startNewParse(fileContents: string, textOffset: number, textLength: number, private _startNewParse(
parseOptions: ParseOptions, diagSink: DiagnosticSink) { fileContents: string,
textOffset: number,
textLength: number,
parseOptions: ParseOptions,
diagSink: DiagnosticSink
) {
this._fileContents = fileContents; this._fileContents = fileContents;
this._parseOptions = parseOptions; this._parseOptions = parseOptions;
this._diagSink = diagSink; this._diagSink = diagSink;
@ -240,8 +318,7 @@ export class Parser {
return this._parseForStatement(asyncToken); return this._parseForStatement(asyncToken);
} }
this._addError('Expected "def", "with" or "for" to follow "async".', this._addError('Expected "def", "with" or "for" to follow "async".', asyncToken);
asyncToken);
return undefined; return undefined;
} }
@ -353,12 +430,13 @@ export class Parser {
let elseSuite: SuiteNode | undefined; let elseSuite: SuiteNode | undefined;
if (!this._consumeTokenIfKeyword(KeywordType.In)) { if (!this._consumeTokenIfKeyword(KeywordType.In)) {
seqExpr = this._handleExpressionParseError( seqExpr = this._handleExpressionParseError(ErrorExpressionCategory.MissingIn, 'Expected "in"');
ErrorExpressionCategory.MissingIn, 'Expected "in"');
forSuite = SuiteNode.create(this._peekToken()); forSuite = SuiteNode.create(this._peekToken());
} else { } else {
seqExpr = this._parseTestListAsExpression( seqExpr = this._parseTestListAsExpression(
ErrorExpressionCategory.MissingExpression, 'Expected expression after "in"'); ErrorExpressionCategory.MissingExpression,
'Expected expression after "in"'
);
forSuite = this._parseLoopSuite(); forSuite = this._parseLoopSuite();
if (this._consumeTokenIfKeyword(KeywordType.Else)) { if (this._consumeTokenIfKeyword(KeywordType.Else)) {
@ -436,16 +514,14 @@ export class Parser {
let seqExpr: ExpressionNode | undefined; let seqExpr: ExpressionNode | undefined;
if (!this._consumeTokenIfKeyword(KeywordType.In)) { if (!this._consumeTokenIfKeyword(KeywordType.In)) {
seqExpr = this._handleExpressionParseError( seqExpr = this._handleExpressionParseError(ErrorExpressionCategory.MissingIn, 'Expected "in"');
ErrorExpressionCategory.MissingIn, 'Expected "in"');
} else { } else {
this._disallowAssignmentExpression(() => { this._disallowAssignmentExpression(() => {
seqExpr = this._parseOrTest(); seqExpr = this._parseOrTest();
}); });
} }
const compForNode = ListComprehensionForNode.create(asyncToken || forToken, const compForNode = ListComprehensionForNode.create(asyncToken || forToken, targetExpr, seqExpr!);
targetExpr, seqExpr!);
if (asyncToken) { if (asyncToken) {
compForNode.isAsync = true; compForNode.isAsync = true;
@ -473,9 +549,7 @@ export class Parser {
private _parseWhileStatement(): WhileNode { private _parseWhileStatement(): WhileNode {
const whileToken = this._getKeywordToken(KeywordType.While); const whileToken = this._getKeywordToken(KeywordType.While);
const whileNode = WhileNode.create(whileToken, const whileNode = WhileNode.create(whileToken, this._parseTestExpression(true), this._parseLoopSuite());
this._parseTestExpression(true),
this._parseLoopSuite());
if (this._consumeTokenIfKeyword(KeywordType.Else)) { if (this._consumeTokenIfKeyword(KeywordType.Else)) {
whileNode.elseSuite = this._parseSuite(); whileNode.elseSuite = this._parseSuite();
@ -533,8 +607,7 @@ export class Parser {
sawCatchAllExcept = true; sawCatchAllExcept = true;
} else { } else {
if (sawCatchAllExcept) { if (sawCatchAllExcept) {
this._addError('A named except clause cannot appear after catch-all except clause', this._addError('A named except clause cannot appear after catch-all except clause', typeExpr);
typeExpr);
} }
} }
@ -575,23 +648,22 @@ export class Parser {
// funcdef: 'def' NAME parameters ['->' test] ':' suite // funcdef: 'def' NAME parameters ['->' test] ':' suite
// parameters: '(' [typedargslist] ')' // parameters: '(' [typedargslist] ')'
private _parseFunctionDef(asyncToken?: KeywordToken, decorators?: DecoratorNode[]): private _parseFunctionDef(asyncToken?: KeywordToken, decorators?: DecoratorNode[]): FunctionNode | ErrorNode {
FunctionNode | ErrorNode {
const defToken = this._getKeywordToken(KeywordType.Def); const defToken = this._getKeywordToken(KeywordType.Def);
const nameToken = this._getTokenIfIdentifier(); const nameToken = this._getTokenIfIdentifier();
if (!nameToken) { if (!nameToken) {
this._addError('Expected function name after "def"', defToken); this._addError('Expected function name after "def"', defToken);
return ErrorNode.create(defToken, return ErrorNode.create(defToken, ErrorExpressionCategory.MissingFunctionParameterList);
ErrorExpressionCategory.MissingFunctionParameterList);
} }
if (!this._consumeTokenIfType(TokenType.OpenParenthesis)) { if (!this._consumeTokenIfType(TokenType.OpenParenthesis)) {
this._addError('Expected "("', this._peekToken()); this._addError('Expected "("', this._peekToken());
return ErrorNode.create(nameToken, return ErrorNode.create(
nameToken,
ErrorExpressionCategory.MissingFunctionParameterList, ErrorExpressionCategory.MissingFunctionParameterList,
NameNode.create(nameToken)); NameNode.create(nameToken)
);
} }
const paramList = this._parseVarArgsList(TokenType.CloseParenthesis, true); const paramList = this._parseVarArgsList(TokenType.CloseParenthesis, true);
@ -807,8 +879,7 @@ export class Parser {
extendRange(paramNode, paramNode.defaultValue); extendRange(paramNode, paramNode.defaultValue);
if (starCount > 0) { if (starCount > 0) {
this._addError(`Parameter with '*' or '**' cannot have default value`, this._addError(`Parameter with '*' or '**' cannot have default value`, paramNode.defaultValue);
paramNode.defaultValue);
} }
} }
@ -913,7 +984,8 @@ export class Parser {
} else { } else {
callNameExpr = ErrorNode.create( callNameExpr = ErrorNode.create(
this._peekToken(), this._peekToken(),
ErrorExpressionCategory.MissingDecoratorCallName); ErrorExpressionCategory.MissingDecoratorCallName
);
} }
break; break;
} }
@ -1004,8 +1076,7 @@ export class Parser {
const breakToken = this._getKeywordToken(KeywordType.Break); const breakToken = this._getKeywordToken(KeywordType.Break);
if (!this._isInLoop) { if (!this._isInLoop) {
this._addError('"break" can be used only within a loop', this._addError('"break" can be used only within a loop', breakToken);
breakToken);
} }
return BreakNode.create(breakToken); return BreakNode.create(breakToken);
@ -1015,11 +1086,9 @@ export class Parser {
const continueToken = this._getKeywordToken(KeywordType.Continue); const continueToken = this._getKeywordToken(KeywordType.Continue);
if (!this._isInLoop) { if (!this._isInLoop) {
this._addError('"continue" can be used only within a loop', this._addError('"continue" can be used only within a loop', continueToken);
continueToken);
} else if (this._isInFinally) { } else if (this._isInFinally) {
this._addError('"continue" cannot be used within a finally clause', this._addError('"continue" cannot be used within a finally clause', continueToken);
continueToken);
} }
return ContinueNode.create(continueToken); return ContinueNode.create(continueToken);
@ -1032,9 +1101,11 @@ export class Parser {
const returnNode = ReturnNode.create(returnToken); const returnNode = ReturnNode.create(returnToken);
if (!this._isNextTokenNeverExpression()) { if (!this._isNextTokenNeverExpression()) {
const returnExpr = this._parseTestOrStarListAsExpression(true, const returnExpr = this._parseTestOrStarListAsExpression(
true,
ErrorExpressionCategory.MissingExpression, ErrorExpressionCategory.MissingExpression,
'Expected expression after "return"'); 'Expected expression after "return"'
);
this._reportConditionalErrorForStarTupleElement(returnExpr); this._reportConditionalErrorForStarTupleElement(returnExpr);
returnNode.returnExpression = returnExpr; returnNode.returnExpression = returnExpr;
returnNode.returnExpression.parent = returnNode; returnNode.returnExpression.parent = returnNode;
@ -1056,9 +1127,8 @@ export class Parser {
// Handle imports from __future__ specially because they can // Handle imports from __future__ specially because they can
// change the way we interpret the rest of the file. // change the way we interpret the rest of the file.
const isFutureImport = modName.leadingDots === 0 && const isFutureImport =
modName.nameParts.length === 1 && modName.leadingDots === 0 && modName.nameParts.length === 1 && modName.nameParts[0].value === '__future__';
modName.nameParts[0].value === '__future__';
const possibleInputToken = this._peekToken(); const possibleInputToken = this._peekToken();
if (!this._consumeTokenIfKeyword(KeywordType.Import)) { if (!this._consumeTokenIfKeyword(KeywordType.Import)) {
@ -1351,18 +1421,18 @@ export class Parser {
const nextToken = this._peekToken(); const nextToken = this._peekToken();
if (this._consumeTokenIfKeyword(KeywordType.From)) { if (this._consumeTokenIfKeyword(KeywordType.From)) {
if (this._getLanguageVersion() < PythonVersion.V33) { if (this._getLanguageVersion() < PythonVersion.V33) {
this._addError( this._addError(`Use of 'yield from' requires Python 3.3 or newer`, nextToken);
`Use of 'yield from' requires Python 3.3 or newer`,
nextToken);
} }
return YieldFromNode.create(yieldToken, this._parseTestExpression(true)); return YieldFromNode.create(yieldToken, this._parseTestExpression(true));
} }
let exprList: ExpressionNode | undefined; let exprList: ExpressionNode | undefined;
if (!this._isNextTokenNeverExpression()) { if (!this._isNextTokenNeverExpression()) {
exprList = this._parseTestOrStarListAsExpression(true, exprList = this._parseTestOrStarListAsExpression(
true,
ErrorExpressionCategory.MissingExpression, ErrorExpressionCategory.MissingExpression,
'Expected expression in yield statement'); 'Expected expression in yield statement'
);
this._reportConditionalErrorForStarTupleElement(exprList); this._reportConditionalErrorForStarTupleElement(exprList);
} }
@ -1417,8 +1487,7 @@ export class Parser {
} }
if (!this._consumeTokenIfType(TokenType.NewLine)) { if (!this._consumeTokenIfType(TokenType.NewLine)) {
this._addError('Statements must be separated by newlines or semicolons', this._addError('Statements must be separated by newlines or semicolons', this._peekToken());
this._peekToken());
} }
return statement; return statement;
@ -1478,8 +1547,8 @@ export class Parser {
// To accommodate empty tuples ("()"), we will reach back to get // To accommodate empty tuples ("()"), we will reach back to get
// the opening parenthesis as the opening token. // the opening parenthesis as the opening token.
const tupleStartRange: TextRange = exprListResult.list.length > 0 ? const tupleStartRange: TextRange =
exprListResult.list[0] : this._peekToken(-1); exprListResult.list.length > 0 ? exprListResult.list[0] : this._peekToken(-1);
const tupleNode = TupleNode.create(tupleStartRange); const tupleNode = TupleNode.create(tupleStartRange);
tupleNode.expressions = exprListResult.list; tupleNode.expressions = exprListResult.list;
@ -1493,9 +1562,7 @@ export class Parser {
return tupleNode; return tupleNode;
} }
private _parseTestListAsExpression(errorCategory: ErrorExpressionCategory, private _parseTestListAsExpression(errorCategory: ErrorExpressionCategory, errorString: string): ExpressionNode {
errorString: string): ExpressionNode {
if (this._isNextTokenNeverExpression()) { if (this._isNextTokenNeverExpression()) {
return this._handleExpressionParseError(errorCategory, errorString); return this._handleExpressionParseError(errorCategory, errorString);
} }
@ -1507,9 +1574,11 @@ export class Parser {
return this._makeExpressionOrTuple(exprListResult); return this._makeExpressionOrTuple(exprListResult);
} }
private _parseTestOrStarListAsExpression(allowAssignmentExpression: boolean, private _parseTestOrStarListAsExpression(
errorCategory: ErrorExpressionCategory, errorString: string): ExpressionNode { allowAssignmentExpression: boolean,
errorCategory: ErrorExpressionCategory,
errorString: string
): ExpressionNode {
if (this._isNextTokenNeverExpression()) { if (this._isNextTokenNeverExpression()) {
return this._handleExpressionParseError(errorCategory, errorString); return this._handleExpressionParseError(errorCategory, errorString);
} }
@ -1531,8 +1600,9 @@ export class Parser {
} }
private _parseTestOrStarExpressionList(allowAssignmentExpression: boolean): ExpressionListResult { private _parseTestOrStarExpressionList(allowAssignmentExpression: boolean): ExpressionListResult {
const exprListResult = this._parseExpressionListGeneric( const exprListResult = this._parseExpressionListGeneric(() =>
() => this._parseTestOrStarExpression(allowAssignmentExpression)); this._parseTestOrStarExpression(allowAssignmentExpression)
);
if (!exprListResult.parseError) { if (!exprListResult.parseError) {
// Make sure that we don't have more than one star expression in the list. // Make sure that we don't have more than one star expression in the list.
@ -1579,8 +1649,7 @@ export class Parser {
return this._parseLambdaExpression(); return this._parseLambdaExpression();
} }
const ifExpr = allowAssignmentExpression ? const ifExpr = allowAssignmentExpression ? this._parseAssignmentExpression() : this._parseOrTest();
this._parseAssignmentExpression() : this._parseOrTest();
if (ifExpr.nodeType === ParseNodeType.Error) { if (ifExpr.nodeType === ParseNodeType.Error) {
return ifExpr; return ifExpr;
} }
@ -1595,8 +1664,7 @@ export class Parser {
} }
if (!this._consumeTokenIfKeyword(KeywordType.Else)) { if (!this._consumeTokenIfKeyword(KeywordType.Else)) {
return this._handleExpressionParseError( return this._handleExpressionParseError(ErrorExpressionCategory.MissingElse, 'Expected "else"');
ErrorExpressionCategory.MissingElse, 'Expected "else"');
} }
const elseExpr = this._parseTestExpression(true); const elseExpr = this._parseTestExpression(true);
@ -1624,15 +1692,11 @@ export class Parser {
} }
if (!this._assignmentExpressionsAllowed) { if (!this._assignmentExpressionsAllowed) {
this._addError( this._addError(`Operator ':=' not allowed in this context`, walrusToken);
`Operator ':=' not allowed in this context`,
walrusToken);
} }
if (this._getLanguageVersion() < PythonVersion.V38) { if (this._getLanguageVersion() < PythonVersion.V38) {
this._addError( this._addError(`Operator ':=' requires Python 3.8 or newer`, walrusToken);
`Operator ':=' requires Python 3.8 or newer`,
walrusToken);
} }
let rightExpr: ExpressionNode; let rightExpr: ExpressionNode;
@ -1710,8 +1774,10 @@ export class Parser {
} }
} else if (this._peekKeywordType() === KeywordType.Not) { } else if (this._peekKeywordType() === KeywordType.Not) {
const tokenAfterNot = this._peekToken(1); const tokenAfterNot = this._peekToken(1);
if (tokenAfterNot.type === TokenType.Keyword && if (
(tokenAfterNot as KeywordToken).keywordType === KeywordType.In) { tokenAfterNot.type === TokenType.Keyword &&
(tokenAfterNot as KeywordToken).keywordType === KeywordType.In
) {
this._getNextToken(); this._getNextToken();
this._getNextToken(); this._getNextToken();
comparisonOperator = OperatorType.NotIn; comparisonOperator = OperatorType.NotIn;
@ -1822,11 +1888,13 @@ export class Parser {
} }
let nextOperator = this._peekOperatorType(); let nextOperator = this._peekOperatorType();
while (nextOperator === OperatorType.Multiply || while (
nextOperator === OperatorType.Multiply ||
nextOperator === OperatorType.MatrixMultiply || nextOperator === OperatorType.MatrixMultiply ||
nextOperator === OperatorType.Divide || nextOperator === OperatorType.Divide ||
nextOperator === OperatorType.Mod || nextOperator === OperatorType.Mod ||
nextOperator === OperatorType.FloorDivide) { nextOperator === OperatorType.FloorDivide
) {
this._getNextToken(); this._getNextToken();
const rightExpr = this._parseArithmeticFactor(); const rightExpr = this._parseArithmeticFactor();
leftExpr = BinaryOperationNode.create(leftExpr, rightExpr, nextOperator); leftExpr = BinaryOperationNode.create(leftExpr, rightExpr, nextOperator);
@ -1841,9 +1909,11 @@ export class Parser {
private _parseArithmeticFactor(): ExpressionNode { private _parseArithmeticFactor(): ExpressionNode {
const nextToken = this._peekToken(); const nextToken = this._peekToken();
const nextOperator = this._peekOperatorType(); const nextOperator = this._peekOperatorType();
if (nextOperator === OperatorType.Add || if (
nextOperator === OperatorType.Add ||
nextOperator === OperatorType.Subtract || nextOperator === OperatorType.Subtract ||
nextOperator === OperatorType.BitwiseInvert) { nextOperator === OperatorType.BitwiseInvert
) {
this._getNextToken(); this._getNextToken();
const expression = this._parseArithmeticFactor(); const expression = this._parseArithmeticFactor();
return UnaryOperationNode.create(nextToken, expression, nextOperator); return UnaryOperationNode.create(nextToken, expression, nextOperator);
@ -1869,9 +1939,7 @@ export class Parser {
if (this._peekKeywordType() === KeywordType.Await && !this._isParsingTypeAnnotation) { if (this._peekKeywordType() === KeywordType.Await && !this._isParsingTypeAnnotation) {
awaitToken = this._getKeywordToken(KeywordType.Await); awaitToken = this._getKeywordToken(KeywordType.Await);
if (this._getLanguageVersion() < PythonVersion.V35) { if (this._getLanguageVersion() < PythonVersion.V35) {
this._addError( this._addError(`Use of 'await' requires Python 3.5 or newer`, awaitToken);
`Use of 'await' requires Python 3.5 or newer`,
awaitToken);
} }
} }
@ -1917,9 +1985,7 @@ export class Parser {
if (atomExpression.nodeType === ParseNodeType.Name && atomExpression.value === 'type') { if (atomExpression.nodeType === ParseNodeType.Name && atomExpression.value === 'type') {
diag.addMessage('Use Type[T] instead'); diag.addMessage('Use Type[T] instead');
} }
this._addError( this._addError(`Function call not allowed in type annotation` + diag.getString(), callNode);
`Function call not allowed in type annotation` + diag.getString(),
callNode);
} }
atomExpression = callNode; atomExpression = callNode;
@ -1930,8 +1996,7 @@ export class Parser {
// type annotations properly. We need to suspend treating strings as // type annotations properly. We need to suspend treating strings as
// type annotations within a Literal subscript. // type annotations within a Literal subscript.
const isLiteralSubscript = const isLiteralSubscript =
(atomExpression.nodeType === ParseNodeType.Name && (atomExpression.nodeType === ParseNodeType.Name && atomExpression.value === 'Literal') ||
atomExpression.value === 'Literal') ||
(atomExpression.nodeType === ParseNodeType.MemberAccess && (atomExpression.nodeType === ParseNodeType.MemberAccess &&
atomExpression.leftExpression.nodeType === ParseNodeType.Name && atomExpression.leftExpression.nodeType === ParseNodeType.Name &&
atomExpression.leftExpression.value === 'typing' && atomExpression.leftExpression.value === 'typing' &&
@ -1960,7 +2025,9 @@ export class Parser {
if (!this._consumeTokenIfType(TokenType.CloseBracket)) { if (!this._consumeTokenIfType(TokenType.CloseBracket)) {
return this._handleExpressionParseError( return this._handleExpressionParseError(
ErrorExpressionCategory.MissingIndexCloseBracket, ErrorExpressionCategory.MissingIndexCloseBracket,
'Expected "]"', indexNode); 'Expected "]"',
indexNode
);
} }
atomExpression = indexNode; atomExpression = indexNode;
@ -1970,10 +2037,11 @@ export class Parser {
if (!memberName) { if (!memberName) {
return this._handleExpressionParseError( return this._handleExpressionParseError(
ErrorExpressionCategory.MissingMemberAccessName, ErrorExpressionCategory.MissingMemberAccessName,
'Expected member name after "."', atomExpression); 'Expected member name after "."',
atomExpression
);
} }
atomExpression = MemberAccessNode.create( atomExpression = MemberAccessNode.create(atomExpression, NameNode.create(memberName));
atomExpression, NameNode.create(memberName));
} else { } else {
break; break;
} }
@ -1988,14 +2056,17 @@ export class Parser {
// subscriptlist: subscript (',' subscript)* [','] // subscriptlist: subscript (',' subscript)* [',']
private _parseSubscriptList(): ExpressionNode { private _parseSubscriptList(): ExpressionNode {
const listResult = this._parseExpressionListGeneric(() => this._parseSubscript(), () => { const listResult = this._parseExpressionListGeneric(
() => this._parseSubscript(),
() => {
// Override the normal terminal check to exclude colons, // Override the normal terminal check to exclude colons,
// which are a valid way to start subscription expressions. // which are a valid way to start subscription expressions.
if (this._peekTokenType() === TokenType.Colon) { if (this._peekTokenType() === TokenType.Colon) {
return false; return false;
} }
return this._isNextTokenNeverExpression(); return this._isNextTokenNeverExpression();
}); }
);
if (listResult.parseError) { if (listResult.parseError) {
return listResult.parseError; return listResult.parseError;
@ -2004,7 +2075,8 @@ export class Parser {
if (listResult.list.length === 0) { if (listResult.list.length === 0) {
return this._handleExpressionParseError( return this._handleExpressionParseError(
ErrorExpressionCategory.MissingIndexOrSlice, ErrorExpressionCategory.MissingIndexOrSlice,
'Expected index or slice expression'); 'Expected index or slice expression'
);
} }
return this._makeExpressionOrTuple(listResult); return this._makeExpressionOrTuple(listResult);
@ -2020,8 +2092,7 @@ export class Parser {
while (true) { while (true) {
const nextTokenType = this._peekTokenType(); const nextTokenType = this._peekTokenType();
if (nextTokenType === TokenType.CloseBracket || if (nextTokenType === TokenType.CloseBracket || nextTokenType === TokenType.Comma) {
nextTokenType === TokenType.Comma) {
break; break;
} }
@ -2069,9 +2140,11 @@ export class Parser {
while (true) { while (true) {
const nextTokenType = this._peekTokenType(); const nextTokenType = this._peekTokenType();
if (nextTokenType === TokenType.CloseParenthesis || if (
nextTokenType === TokenType.CloseParenthesis ||
nextTokenType === TokenType.NewLine || nextTokenType === TokenType.NewLine ||
nextTokenType === TokenType.EndOfStream) { nextTokenType === TokenType.EndOfStream
) {
break; break;
} }
@ -2079,9 +2152,7 @@ export class Parser {
if (arg.name) { if (arg.name) {
sawKeywordArg = true; sawKeywordArg = true;
} else if (sawKeywordArg && arg.argumentCategory === ArgumentCategory.Simple) { } else if (sawKeywordArg && arg.argumentCategory === ArgumentCategory.Simple) {
this._addError( this._addError('Positional argument cannot appear after named arguments', arg);
'Positional argument cannot appear after named arguments',
arg);
} }
argList.push(arg); argList.push(arg);
@ -2151,9 +2222,7 @@ export class Parser {
if (nextToken.type === TokenType.Number) { if (nextToken.type === TokenType.Number) {
const numberNode = NumberNode.create(this._getNextToken() as NumberToken); const numberNode = NumberNode.create(this._getNextToken() as NumberToken);
if (this._isParsingTypeAnnotation) { if (this._isParsingTypeAnnotation) {
this._addError( this._addError('Numeric literal not allowed in type annotation', numberNode);
'Numeric literal not allowed in type annotation',
numberNode);
} }
return numberNode; return numberNode;
} }
@ -2174,10 +2243,13 @@ export class Parser {
// and emit an error. // and emit an error.
this._addError( this._addError(
'Expressions surrounded by backticks are not supported in Python 3.x; use repr instead', 'Expressions surrounded by backticks are not supported in Python 3.x; use repr instead',
nextToken); nextToken
);
const expressionNode = this._parseTestListAsExpression(ErrorExpressionCategory.MissingExpression, const expressionNode = this._parseTestListAsExpression(
'Expected expression'); ErrorExpressionCategory.MissingExpression,
'Expected expression'
);
this._consumeTokenIfType(TokenType.Backtick); this._consumeTokenIfType(TokenType.Backtick);
return expressionNode; return expressionNode;
@ -2191,9 +2263,7 @@ export class Parser {
// a zero-length tuple. // a zero-length tuple.
const diag = new DiagnosticAddendum(); const diag = new DiagnosticAddendum();
diag.addMessage('Use Tuple[T1, ..., Tn] instead'); diag.addMessage('Use Tuple[T1, ..., Tn] instead');
this._addError( this._addError('Tuple expression not allowed in type annotation' + diag.getString(), tupleNode);
'Tuple expression not allowed in type annotation' + diag.getString(),
tupleNode);
} }
return tupleNode; return tupleNode;
} else if (nextToken.type === TokenType.OpenBracket) { } else if (nextToken.type === TokenType.OpenBracket) {
@ -2201,9 +2271,7 @@ export class Parser {
if (this._isParsingTypeAnnotation && !this._isParsingIndexTrailer) { if (this._isParsingTypeAnnotation && !this._isParsingIndexTrailer) {
const diag = new DiagnosticAddendum(); const diag = new DiagnosticAddendum();
diag.addMessage('Use List[T] instead'); diag.addMessage('Use List[T] instead');
this._addError( this._addError('List expression not allowed in type annotation' + diag.getString(), listNode);
'List expression not allowed in type annotation' + diag.getString(),
listNode);
} }
return listNode; return listNode;
} else if (nextToken.type === TokenType.OpenCurlyBrace) { } else if (nextToken.type === TokenType.OpenCurlyBrace) {
@ -2211,27 +2279,24 @@ export class Parser {
if (this._isParsingTypeAnnotation) { if (this._isParsingTypeAnnotation) {
const diag = new DiagnosticAddendum(); const diag = new DiagnosticAddendum();
diag.addMessage('Use Dict[T1, T2] instead'); diag.addMessage('Use Dict[T1, T2] instead');
this._addError( this._addError('Dictionary expression not allowed in type annotation' + diag.getString(), dictNode);
'Dictionary expression not allowed in type annotation' + diag.getString(),
dictNode);
} }
return dictNode; return dictNode;
} }
if (nextToken.type === TokenType.Keyword) { if (nextToken.type === TokenType.Keyword) {
const keywordToken = nextToken as KeywordToken; const keywordToken = nextToken as KeywordToken;
if (keywordToken.keywordType === KeywordType.False || if (
keywordToken.keywordType === KeywordType.False ||
keywordToken.keywordType === KeywordType.True || keywordToken.keywordType === KeywordType.True ||
keywordToken.keywordType === KeywordType.Debug || keywordToken.keywordType === KeywordType.Debug ||
keywordToken.keywordType === KeywordType.None) { keywordToken.keywordType === KeywordType.None
) {
const constNode = ConstantNode.create(this._getNextToken() as KeywordToken); const constNode = ConstantNode.create(this._getNextToken() as KeywordToken);
if (this._isParsingTypeAnnotation) { if (this._isParsingTypeAnnotation) {
if (keywordToken.keywordType !== KeywordType.None) { if (keywordToken.keywordType !== KeywordType.None) {
this._addError( this._addError('Keyword not allowed in type annotation', constNode);
'Keyword not allowed in type annotation',
constNode);
} }
} }
@ -2245,18 +2310,18 @@ export class Parser {
} }
} }
return this._handleExpressionParseError( return this._handleExpressionParseError(ErrorExpressionCategory.MissingExpression, 'Expected expression');
ErrorExpressionCategory.MissingExpression,
'Expected expression');
} }
// Allocates a dummy "error expression" and consumes the remainder // Allocates a dummy "error expression" and consumes the remainder
// of the tokens on the line for error recovery. A partially-completed // of the tokens on the line for error recovery. A partially-completed
// child node can be passed to help the completion provider determine // child node can be passed to help the completion provider determine
// what to do. // what to do.
private _handleExpressionParseError(category: ErrorExpressionCategory, private _handleExpressionParseError(
errorMsg: string, childNode?: ExpressionNode): ErrorNode { category: ErrorExpressionCategory,
errorMsg: string,
childNode?: ExpressionNode
): ErrorNode {
this._addError(errorMsg, this._peekToken()); this._addError(errorMsg, this._peekToken());
const expr = ErrorNode.create(this._peekToken(), category, childNode); const expr = ErrorNode.create(this._peekToken(), category, childNode);
this._consumeTokensUntilType(TokenType.NewLine); this._consumeTokensUntilType(TokenType.NewLine);
@ -2305,9 +2370,7 @@ export class Parser {
const yieldExpr = this._tryParseYieldExpression(); const yieldExpr = this._tryParseYieldExpression();
if (yieldExpr) { if (yieldExpr) {
if (this._peekTokenType() !== TokenType.CloseParenthesis) { if (this._peekTokenType() !== TokenType.CloseParenthesis) {
return this._handleExpressionParseError( return this._handleExpressionParseError(ErrorExpressionCategory.MissingTupleCloseParen, 'Expected ")"');
ErrorExpressionCategory.MissingTupleCloseParen,
'Expected ")"');
} else { } else {
extendRange(yieldExpr, this._getNextToken()); extendRange(yieldExpr, this._getNextToken());
} }
@ -2319,9 +2382,7 @@ export class Parser {
const tupleOrExpression = this._makeExpressionOrTuple(exprListResult); const tupleOrExpression = this._makeExpressionOrTuple(exprListResult);
if (this._peekTokenType() !== TokenType.CloseParenthesis) { if (this._peekTokenType() !== TokenType.CloseParenthesis) {
return this._handleExpressionParseError( return this._handleExpressionParseError(ErrorExpressionCategory.MissingTupleCloseParen, 'Expected ")"');
ErrorExpressionCategory.MissingTupleCloseParen,
'Expected ")"');
} else { } else {
extendRange(tupleOrExpression, this._getNextToken()); extendRange(tupleOrExpression, this._getNextToken());
} }
@ -2338,9 +2399,7 @@ export class Parser {
const exprListResult = this._parseTestListWithComprehension(); const exprListResult = this._parseTestListWithComprehension();
const closeBracket: Token | undefined = this._peekToken(); const closeBracket: Token | undefined = this._peekToken();
if (!this._consumeTokenIfType(TokenType.CloseBracket)) { if (!this._consumeTokenIfType(TokenType.CloseBracket)) {
return this._handleExpressionParseError( return this._handleExpressionParseError(ErrorExpressionCategory.MissingListCloseBracket, 'Expected "]"');
ErrorExpressionCategory.MissingListCloseBracket,
'Expected "]"');
} }
const listAtom = ListNode.create(startBracket); const listAtom = ListNode.create(startBracket);
@ -2358,7 +2417,8 @@ export class Parser {
private _parseTestListWithComprehension(): ExpressionListResult { private _parseTestListWithComprehension(): ExpressionListResult {
let sawComprehension = false; let sawComprehension = false;
return this._parseExpressionListGeneric(() => { return this._parseExpressionListGeneric(
() => {
let expr = this._parseTestOrStarExpression(true); let expr = this._parseTestOrStarExpression(true);
const listComp = this._tryParseListComprehension(expr); const listComp = this._tryParseListComprehension(expr);
if (listComp) { if (listComp) {
@ -2368,7 +2428,8 @@ export class Parser {
return expr; return expr;
}, },
() => this._isNextTokenNeverExpression(), () => this._isNextTokenNeverExpression(),
() => sawComprehension); () => sawComprehension
);
} }
// '{' [dictorsetmaker] '}' // '{' [dictorsetmaker] '}'
@ -2501,11 +2562,11 @@ export class Parser {
return dictionaryAtom; return dictionaryAtom;
} }
private _parseExpressionListGeneric(parser: () => ExpressionNode, private _parseExpressionListGeneric(
parser: () => ExpressionNode,
terminalCheck: () => boolean = () => this._isNextTokenNeverExpression(), terminalCheck: () => boolean = () => this._isNextTokenNeverExpression(),
finalEntryCheck: () => boolean = () => false): finalEntryCheck: () => boolean = () => false
ExpressionListResult { ): ExpressionListResult {
let trailingComma = false; let trailingComma = false;
const list: ExpressionNode[] = []; const list: ExpressionNode[] = [];
let parseError: ErrorNode | undefined; let parseError: ErrorNode | undefined;
@ -2545,8 +2606,11 @@ export class Parser {
// augassign: ('+=' | '-=' | '*=' | '@=' | '/=' | '%=' | '&=' | '|=' | '^=' | // augassign: ('+=' | '-=' | '*=' | '@=' | '/=' | '%=' | '&=' | '|=' | '^=' |
// '<<=' | '>>=' | '**=' | '//=') // '<<=' | '>>=' | '**=' | '//=')
private _parseExpressionStatement(): ExpressionNode { private _parseExpressionStatement(): ExpressionNode {
let leftExpr = this._parseTestOrStarListAsExpression(false, let leftExpr = this._parseTestOrStarListAsExpression(
ErrorExpressionCategory.MissingExpression, 'Expected expression'); false,
ErrorExpressionCategory.MissingExpression,
'Expected expression'
);
let annotationExpr: ExpressionNode | undefined; let annotationExpr: ExpressionNode | undefined;
if (leftExpr.nodeType === ParseNodeType.Error) { if (leftExpr.nodeType === ParseNodeType.Error) {
@ -2560,8 +2624,7 @@ export class Parser {
leftExpr = TypeAnnotationNode.create(leftExpr, annotationExpr); leftExpr = TypeAnnotationNode.create(leftExpr, annotationExpr);
if (!this._parseOptions.isStubFile && this._getLanguageVersion() < PythonVersion.V36) { if (!this._parseOptions.isStubFile && this._getLanguageVersion() < PythonVersion.V36) {
this._addError('Type annotations for variables requires Python 3.6 or newer', this._addError('Type annotations for variables requires Python 3.6 or newer', annotationExpr);
annotationExpr);
} }
}); });
@ -2581,17 +2644,18 @@ export class Parser {
if (!annotationExpr && Tokenizer.isOperatorAssignment(this._peekOperatorType())) { if (!annotationExpr && Tokenizer.isOperatorAssignment(this._peekOperatorType())) {
const operatorToken = this._getNextToken() as OperatorToken; const operatorToken = this._getNextToken() as OperatorToken;
const rightExpr = this._tryParseYieldExpression() || const rightExpr =
this._tryParseYieldExpression() ||
this._parseTestListAsExpression( this._parseTestListAsExpression(
ErrorExpressionCategory.MissingExpression, ErrorExpressionCategory.MissingExpression,
'Expected expression to the right of operator'); 'Expected expression to the right of operator'
);
// Make a shallow copy of the dest expression but give it a new ID. // Make a shallow copy of the dest expression but give it a new ID.
const destExpr = Object.assign({}, leftExpr); const destExpr = Object.assign({}, leftExpr);
destExpr.id = getNextNodeId(); destExpr.id = getNextNodeId();
return AugmentedAssignmentNode.create(leftExpr, rightExpr, return AugmentedAssignmentNode.create(leftExpr, rightExpr, operatorToken.operatorType, destExpr);
operatorToken.operatorType, destExpr);
} }
return leftExpr; return leftExpr;
@ -2601,9 +2665,11 @@ export class Parser {
let rightExpr: ExpressionNode | undefined; let rightExpr: ExpressionNode | undefined;
rightExpr = this._tryParseYieldExpression(); rightExpr = this._tryParseYieldExpression();
if (!rightExpr) { if (!rightExpr) {
rightExpr = this._parseTestOrStarListAsExpression(false, rightExpr = this._parseTestOrStarListAsExpression(
false,
ErrorExpressionCategory.MissingExpression, ErrorExpressionCategory.MissingExpression,
'Expected expression to the right of "="'); 'Expected expression to the right of "="'
);
} }
if (rightExpr.nodeType === ParseNodeType.Error) { if (rightExpr.nodeType === ParseNodeType.Error) {
@ -2683,8 +2749,7 @@ export class Parser {
return undefined; return undefined;
} }
const interTokenContents = this._fileContents!.substring( const interTokenContents = this._fileContents!.substring(curToken.start + curToken.length, nextToken.start);
curToken.start + curToken.length, nextToken.start);
const commentRegEx = /^(\s*#\s*type:\s*)([^\r\n]*)/; const commentRegEx = /^(\s*#\s*type:\s*)([^\r\n]*)/;
const match = interTokenContents.match(commentRegEx); const match = interTokenContents.match(commentRegEx);
if (!match) { if (!match) {
@ -2700,14 +2765,25 @@ export class Parser {
} }
const tokenOffset = curToken.start + curToken.length + match[1].length; const tokenOffset = curToken.start + curToken.length + match[1].length;
const stringToken = StringToken.create(tokenOffset, const stringToken = StringToken.create(
typeString.length, StringTokenFlags.None, typeString, 0, undefined); tokenOffset,
typeString.length,
StringTokenFlags.None,
typeString,
0,
undefined
);
const stringNode = this._makeStringNode(stringToken); const stringNode = this._makeStringNode(stringToken);
const stringListNode = StringListNode.create([stringNode]); const stringListNode = StringListNode.create([stringNode]);
const parser = new Parser(); const parser = new Parser();
const parseResults = parser.parseTextExpression(this._fileContents!, const parseResults = parser.parseTextExpression(
tokenOffset, typeString.length, this._parseOptions, true); this._fileContents!,
tokenOffset,
typeString.length,
this._parseOptions,
true
);
parseResults.diagnostics.forEach(diag => { parseResults.diagnostics.forEach(diag => {
this._addError(diag.message, stringListNode); this._addError(diag.message, stringListNode);
@ -2734,17 +2810,23 @@ export class Parser {
// contains formatting directives that start with a ! or :. // contains formatting directives that start with a ! or :.
const segmentExprLength = this._getFormatStringExpressionLength(segment.value); const segmentExprLength = this._getFormatStringExpressionLength(segment.value);
const parseResults = parser.parseTextExpression(this._fileContents!, const parseResults = parser.parseTextExpression(
stringToken.start + stringToken.prefixLength + stringToken.quoteMarkLength + this._fileContents!,
segment.offset, segmentExprLength, this._parseOptions, false); stringToken.start + stringToken.prefixLength + stringToken.quoteMarkLength + segment.offset,
segmentExprLength,
this._parseOptions,
false
);
parseResults.diagnostics.forEach(diag => { parseResults.diagnostics.forEach(diag => {
const textRangeStart = (diag.range ? const textRangeStart =
convertPositionToOffset(diag.range.start, parseResults.lines) : (diag.range
stringToken.start) || stringToken.start; ? convertPositionToOffset(diag.range.start, parseResults.lines)
const textRangeEnd = (diag.range ? : stringToken.start) || stringToken.start;
(convertPositionToOffset(diag.range.end, parseResults.lines) || 0) + 1 : const textRangeEnd =
stringToken.start + stringToken.length) || (stringToken.start + stringToken.length); (diag.range
? (convertPositionToOffset(diag.range.end, parseResults.lines) || 0) + 1
: stringToken.start + stringToken.length) || stringToken.start + stringToken.length;
const textRange = { start: textRangeStart, length: textRangeEnd - textRangeStart }; const textRange = { start: textRangeStart, length: textRangeEnd - textRangeStart };
this._addError(diag.message, textRange); this._addError(diag.message, textRange);
}); });
@ -2755,8 +2837,12 @@ export class Parser {
} }
} }
return FormatStringNode.create(stringToken, unescapedResult.value, return FormatStringNode.create(
unescapedResult.unescapeErrors.length > 0, formatExpressions); stringToken,
unescapedResult.value,
unescapedResult.unescapeErrors.length > 0,
formatExpressions
);
} }
private _getFormatStringExpressionLength(segmentValue: string): number { private _getFormatStringExpressionLength(segmentValue: string): number {
@ -2774,8 +2860,8 @@ export class Parser {
while (segmentExprLength < segmentValue.length) { while (segmentExprLength < segmentValue.length) {
const curChar = segmentValue[segmentExprLength]; const curChar = segmentValue[segmentExprLength];
const ignoreSeparator = inSingleQuote || inDoubleQuote || braceCount > 0 || const ignoreSeparator =
parenCount > 0 || bracketCount > 0; inSingleQuote || inDoubleQuote || braceCount > 0 || parenCount > 0 || bracketCount > 0;
const inString = inSingleQuote || inDoubleQuote; const inString = inSingleQuote || inDoubleQuote;
if (curChar === '=') { if (curChar === '=') {
@ -2788,12 +2874,14 @@ export class Parser {
} else if (curChar === '!') { } else if (curChar === '!') {
if (!ignoreSeparator) { if (!ignoreSeparator) {
// Allow !=, as per PEP 498 // Allow !=, as per PEP 498
if (segmentExprLength === segmentValue.length - 1 || if (
segmentValue[segmentExprLength + 1] !== '=') { segmentExprLength === segmentValue.length - 1 ||
segmentValue[segmentExprLength + 1] !== '='
) {
break; break;
} }
} }
} else if (curChar === '\'') { } else if (curChar === "'") {
if (!inDoubleQuote) { if (!inDoubleQuote) {
inSingleQuote = !inSingleQuote; inSingleQuote = !inSingleQuote;
} }
@ -2879,8 +2967,13 @@ export class Parser {
this._addError('Type hints cannot contain escape characters', stringNode); this._addError('Type hints cannot contain escape characters', stringNode);
} else { } else {
const parser = new Parser(); const parser = new Parser();
const parseResults = parser.parseTextExpression(this._fileContents!, const parseResults = parser.parseTextExpression(
tokenOffset + prefixLength, unescapedString.length, this._parseOptions, true); this._fileContents!,
tokenOffset + prefixLength,
unescapedString.length,
this._parseOptions,
true
);
parseResults.diagnostics.forEach(diag => { parseResults.diagnostics.forEach(diag => {
this._addError(diag.message, stringNode); this._addError(diag.message, stringNode);
@ -2997,8 +3090,7 @@ export class Parser {
} }
if (this._tokenIndex + count >= this._tokenizerOutput!.tokens.count) { if (this._tokenIndex + count >= this._tokenizerOutput!.tokens.count) {
return this._tokenizerOutput!.tokens.getItemAt( return this._tokenizerOutput!.tokens.getItemAt(this._tokenizerOutput!.tokens.count - 1);
this._tokenizerOutput!.tokens.count - 1);
} }
return this._tokenizerOutput!.tokens.getItemAt(this._tokenIndex + count); return this._tokenizerOutput!.tokens.getItemAt(this._tokenIndex + count);
@ -3036,8 +3128,7 @@ export class Parser {
if (nextToken.type === TokenType.Invalid) { if (nextToken.type === TokenType.Invalid) {
this._getNextToken(); this._getNextToken();
this._addError(`Invalid character in identifier`, nextToken); this._addError(`Invalid character in identifier`, nextToken);
return IdentifierToken.create(nextToken.start, return IdentifierToken.create(nextToken.start, nextToken.length, '', nextToken.comments);
nextToken.length, '', nextToken.comments);
} }
// If keywords are allowed in this context, convert the keyword // If keywords are allowed in this context, convert the keyword
@ -3047,8 +3138,7 @@ export class Parser {
if (!disallowedKeywords.find(type => type === keywordType)) { if (!disallowedKeywords.find(type => type === keywordType)) {
const keywordText = this._fileContents!.substr(nextToken.start, nextToken.length); const keywordText = this._fileContents!.substr(nextToken.start, nextToken.length);
this._getNextToken(); this._getNextToken();
return IdentifierToken.create(nextToken.start, return IdentifierToken.create(nextToken.start, nextToken.length, keywordText, nextToken.comments);
nextToken.length, keywordText, nextToken.comments);
} }
} }
@ -3113,8 +3203,9 @@ export class Parser {
private _addError(message: string, range: TextRange) { private _addError(message: string, range: TextRange) {
assert(range !== undefined); assert(range !== undefined);
this._diagSink.addError(message, this._diagSink.addError(
convertOffsetsToRange(range.start, range.start + range.length, message,
this._tokenizerOutput!.lines)); convertOffsetsToRange(range.start, range.start + range.length, this._tokenizerOutput!.lines)
);
} }
} }

2
server/src/parser/stringTokenUtils.ts
View File

@ -124,7 +124,7 @@ export function getUnescapedString(stringToken: StringToken): UnescapedString {
const char = String.fromCharCode(charCode); const char = String.fromCharCode(charCode);
output.value += char; output.value += char;
formatSegment.value += char; formatSegment.value += char;
} };
while (true) { while (true) {
let curChar = getEscapedCharacter(); let curChar = getEscapedCharacter();

243
server/src/parser/tokenizer.ts
View File

@ -16,47 +16,62 @@ import { TextRange } from '../common/textRange';
import { TextRangeCollection } from '../common/textRangeCollection'; import { TextRangeCollection } from '../common/textRangeCollection';
import { isBinary, isDecimal, isHex, isIdentifierChar, isIdentifierStartChar, isOctal } from './characters'; import { isBinary, isDecimal, isHex, isIdentifierChar, isIdentifierStartChar, isOctal } from './characters';
import { CharacterStream } from './characterStream'; import { CharacterStream } from './characterStream';
import { Comment, DedentToken, IdentifierToken, IndentToken, KeywordToken, import {
KeywordType, NewLineToken, NewLineType, NumberToken, OperatorFlags, OperatorToken, Comment,
OperatorType, StringToken, StringTokenFlags, Token, TokenType } from './tokenizerTypes'; DedentToken,
IdentifierToken,
IndentToken,
KeywordToken,
KeywordType,
NewLineToken,
NewLineType,
NumberToken,
OperatorFlags,
OperatorToken,
OperatorType,
StringToken,
StringTokenFlags,
Token,
TokenType
} from './tokenizerTypes';
const _keywords: { [key: string]: KeywordType } = { const _keywords: { [key: string]: KeywordType } = {
'and': KeywordType.And, and: KeywordType.And,
'as': KeywordType.As, as: KeywordType.As,
'assert': KeywordType.Assert, assert: KeywordType.Assert,
'async': KeywordType.Async, async: KeywordType.Async,
'await': KeywordType.Await, await: KeywordType.Await,
'break': KeywordType.Break, break: KeywordType.Break,
'class': KeywordType.Class, class: KeywordType.Class,
'continue': KeywordType.Continue, continue: KeywordType.Continue,
'__debug__': KeywordType.Debug, __debug__: KeywordType.Debug,
'def': KeywordType.Def, def: KeywordType.Def,
'del': KeywordType.Del, del: KeywordType.Del,
'elif': KeywordType.Elif, elif: KeywordType.Elif,
'else': KeywordType.Else, else: KeywordType.Else,
'except': KeywordType.Except, except: KeywordType.Except,
'finally': KeywordType.Finally, finally: KeywordType.Finally,
'for': KeywordType.For, for: KeywordType.For,
'from': KeywordType.From, from: KeywordType.From,
'global': KeywordType.Global, global: KeywordType.Global,
'if': KeywordType.If, if: KeywordType.If,
'import': KeywordType.Import, import: KeywordType.Import,
'in': KeywordType.In, in: KeywordType.In,
'is': KeywordType.Is, is: KeywordType.Is,
'lambda': KeywordType.Lambda, lambda: KeywordType.Lambda,
'nonlocal': KeywordType.Nonlocal, nonlocal: KeywordType.Nonlocal,
'not': KeywordType.Not, not: KeywordType.Not,
'or': KeywordType.Or, or: KeywordType.Or,
'pass': KeywordType.Pass, pass: KeywordType.Pass,
'raise': KeywordType.Raise, raise: KeywordType.Raise,
'return': KeywordType.Return, return: KeywordType.Return,
'try': KeywordType.Try, try: KeywordType.Try,
'while': KeywordType.While, while: KeywordType.While,
'with': KeywordType.With, with: KeywordType.With,
'yield': KeywordType.Yield, yield: KeywordType.Yield,
'False': KeywordType.False, False: KeywordType.False,
'None': KeywordType.None, None: KeywordType.None,
'True': KeywordType.True True: KeywordType.True
}; };
const _operatorInfo: { [key: number]: OperatorFlags } = { const _operatorInfo: { [key: number]: OperatorFlags } = {
@ -104,7 +119,7 @@ const _operatorInfo: { [key: number]: OperatorFlags } = {
[OperatorType.NotIn]: OperatorFlags.Binary [OperatorType.NotIn]: OperatorFlags.Binary
}; };
const _byteOrderMarker = 0xFEFF; const _byteOrderMarker = 0xfeff;
export interface TokenizerOutput { export interface TokenizerOutput {
// List of all tokens. // List of all tokens.
@ -229,8 +244,7 @@ export class Tokenizer {
} else if (this._indentCount > 0) { } else if (this._indentCount > 0) {
// Compute the average number of spaces per indent // Compute the average number of spaces per indent
// to estimate the predominant tab value. // to estimate the predominant tab value.
let averageSpacePerIndent = Math.round( let averageSpacePerIndent = Math.round(this._indentSpacesTotal / this._indentCount);
this._indentSpacesTotal / this._indentCount);
if (averageSpacePerIndent < 1) { if (averageSpacePerIndent < 1) {
averageSpacePerIndent = 1; averageSpacePerIndent = 1;
} else if (averageSpacePerIndent > 8) { } else if (averageSpacePerIndent > 8) {
@ -249,7 +263,7 @@ export class Tokenizer {
typeIgnoreAll: this._typeIgnoreAll, typeIgnoreAll: this._typeIgnoreAll,
predominantEndOfLineSequence, predominantEndOfLineSequence,
predominantTabSequence, predominantTabSequence,
predominantSingleQuoteCharacter: this._singleQuoteCount >= this._doubleQuoteCount ? '\'' : '"' predominantSingleQuoteCharacter: this._singleQuoteCount >= this._doubleQuoteCount ? "'" : '"'
}; };
} }
@ -319,8 +333,7 @@ export class Tokenizer {
case Char.CarriageReturn: { case Char.CarriageReturn: {
const length = this._cs.nextChar === Char.LineFeed ? 2 : 1; const length = this._cs.nextChar === Char.LineFeed ? 2 : 1;
const newLineType = length === 2 ? const newLineType = length === 2 ? NewLineType.CarriageReturnLineFeed : NewLineType.CarriageReturn;
NewLineType.CarriageReturnLineFeed : NewLineType.CarriageReturn;
this._handleNewLine(length, newLineType); this._handleNewLine(length, newLineType);
return true; return true;
} }
@ -350,8 +363,7 @@ export class Tokenizer {
case Char.OpenParenthesis: { case Char.OpenParenthesis: {
this._parenDepth++; this._parenDepth++;
this._tokens.push(Token.create(TokenType.OpenParenthesis, this._tokens.push(Token.create(TokenType.OpenParenthesis, this._cs.position, 1, this._getComments()));
this._cs.position, 1, this._getComments()));
break; break;
} }
@ -359,15 +371,13 @@ export class Tokenizer {
if (this._parenDepth > 0) { if (this._parenDepth > 0) {
this._parenDepth--; this._parenDepth--;
} }
this._tokens.push(Token.create(TokenType.CloseParenthesis, this._tokens.push(Token.create(TokenType.CloseParenthesis, this._cs.position, 1, this._getComments()));
this._cs.position, 1, this._getComments()));
break; break;
} }
case Char.OpenBracket: { case Char.OpenBracket: {
this._parenDepth++; this._parenDepth++;
this._tokens.push(Token.create(TokenType.OpenBracket, this._tokens.push(Token.create(TokenType.OpenBracket, this._cs.position, 1, this._getComments()));
this._cs.position, 1, this._getComments()));
break; break;
} }
@ -375,15 +385,13 @@ export class Tokenizer {
if (this._parenDepth > 0) { if (this._parenDepth > 0) {
this._parenDepth--; this._parenDepth--;
} }
this._tokens.push(Token.create(TokenType.CloseBracket, this._tokens.push(Token.create(TokenType.CloseBracket, this._cs.position, 1, this._getComments()));
this._cs.position, 1, this._getComments()));
break; break;
} }
case Char.OpenBrace: { case Char.OpenBrace: {
this._parenDepth++; this._parenDepth++;
this._tokens.push(Token.create(TokenType.OpenCurlyBrace, this._tokens.push(Token.create(TokenType.OpenCurlyBrace, this._cs.position, 1, this._getComments()));
this._cs.position, 1, this._getComments()));
break; break;
} }
@ -391,38 +399,34 @@ export class Tokenizer {
if (this._parenDepth > 0) { if (this._parenDepth > 0) {
this._parenDepth--; this._parenDepth--;
} }
this._tokens.push(Token.create(TokenType.CloseCurlyBrace, this._tokens.push(Token.create(TokenType.CloseCurlyBrace, this._cs.position, 1, this._getComments()));
this._cs.position, 1, this._getComments()));
break; break;
} }
case Char.Comma: { case Char.Comma: {
this._tokens.push(Token.create(TokenType.Comma, this._tokens.push(Token.create(TokenType.Comma, this._cs.position, 1, this._getComments()));
this._cs.position, 1, this._getComments()));
break; break;
} }
case Char.Backtick: { case Char.Backtick: {
this._tokens.push(Token.create(TokenType.Backtick, this._tokens.push(Token.create(TokenType.Backtick, this._cs.position, 1, this._getComments()));
this._cs.position, 1, this._getComments()));
break; break;
} }
case Char.Semicolon: { case Char.Semicolon: {
this._tokens.push(Token.create(TokenType.Semicolon, this._tokens.push(Token.create(TokenType.Semicolon, this._cs.position, 1, this._getComments()));
this._cs.position, 1, this._getComments()));
break; break;
} }
case Char.Colon: { case Char.Colon: {
if (this._cs.nextChar === Char.Equal) { if (this._cs.nextChar === Char.Equal) {
this._tokens.push(OperatorToken.create(this._cs.position, this._tokens.push(
2, OperatorType.Walrus, this._getComments())); OperatorToken.create(this._cs.position, 2, OperatorType.Walrus, this._getComments())
);
this._cs.advance(1); this._cs.advance(1);
break; break;
} }
this._tokens.push(Token.create(TokenType.Colon, this._tokens.push(Token.create(TokenType.Colon, this._cs.position, 1, this._getComments()));
this._cs.position, 1, this._getComments()));
break; break;
} }
@ -435,13 +439,11 @@ export class Tokenizer {
if (this._cs.currentChar === Char.Period) { if (this._cs.currentChar === Char.Period) {
if (this._cs.nextChar === Char.Period && this._cs.lookAhead(2) === Char.Period) { if (this._cs.nextChar === Char.Period && this._cs.lookAhead(2) === Char.Period) {
this._tokens.push(Token.create(TokenType.Ellipsis, this._tokens.push(Token.create(TokenType.Ellipsis, this._cs.position, 3, this._getComments()));
this._cs.position, 3, this._getComments()));
this._cs.advance(3); this._cs.advance(3);
return true; return true;
} }
this._tokens.push(Token.create(TokenType.Dot, this._tokens.push(Token.create(TokenType.Dot, this._cs.position, 1, this._getComments()));
this._cs.position, 1, this._getComments()));
break; break;
} }
@ -471,8 +473,7 @@ export class Tokenizer {
// New lines are ignored within parentheses. // New lines are ignored within parentheses.
// We'll also avoid adding multiple newlines in a row to simplify parsing. // We'll also avoid adding multiple newlines in a row to simplify parsing.
if (this._tokens.length === 0 || this._tokens[this._tokens.length - 1].type !== TokenType.NewLine) { if (this._tokens.length === 0 || this._tokens[this._tokens.length - 1].type !== TokenType.NewLine) {
this._tokens.push(NewLineToken.create(this._cs.position, this._tokens.push(NewLineToken.create(this._cs.position, length, newLineType, this._getComments()));
length, newLineType, this._getComments()));
} }
} }
if (newLineType === NewLineType.CarriageReturn) { if (newLineType === NewLineType.CarriageReturn) {
@ -539,8 +540,7 @@ export class Tokenizer {
this._indentSpacesTotal += spaceCount; this._indentSpacesTotal += spaceCount;
this._indentAmounts.push(spaceCount); this._indentAmounts.push(spaceCount);
this._tokens.push(IndentToken.create(this._cs.position, 0, this._tokens.push(IndentToken.create(this._cs.position, 0, spaceCount, this._getComments()));
spaceCount, this._getComments()));
} }
} else { } else {
if (this._indentAmounts[this._indentAmounts.length - 1] < spaceCount) { if (this._indentAmounts[this._indentAmounts.length - 1] < spaceCount) {
@ -551,28 +551,29 @@ export class Tokenizer {
this._indentSpacesTotal += spaceCount - this._indentAmounts[this._indentAmounts.length - 1]; this._indentSpacesTotal += spaceCount - this._indentAmounts[this._indentAmounts.length - 1];
this._indentAmounts.push(spaceCount); this._indentAmounts.push(spaceCount);
this._tokens.push(IndentToken.create(this._cs.position, 0, this._tokens.push(IndentToken.create(this._cs.position, 0, spaceCount, this._getComments()));
spaceCount, this._getComments()));
} else { } else {
// The Python spec says that dedent amounts need to match the indent // The Python spec says that dedent amounts need to match the indent
// amount exactly. An error is generated at runtime if it doesn't. // amount exactly. An error is generated at runtime if it doesn't.
// We'll record that error condition within the token, allowing the // We'll record that error condition within the token, allowing the
// parser to report it later. // parser to report it later.
const dedentPoints: number[] = []; const dedentPoints: number[] = [];
while (this._indentAmounts.length > 0 && while (
this._indentAmounts[this._indentAmounts.length - 1] > spaceCount) { this._indentAmounts.length > 0 &&
dedentPoints.push(this._indentAmounts.length > 1 ? this._indentAmounts[this._indentAmounts.length - 1] > spaceCount
this._indentAmounts[this._indentAmounts.length - 2] : 0); ) {
dedentPoints.push(
this._indentAmounts.length > 1 ? this._indentAmounts[this._indentAmounts.length - 2] : 0
);
this._indentAmounts.pop(); this._indentAmounts.pop();
} }
dedentPoints.forEach((dedentAmount, index) => { dedentPoints.forEach((dedentAmount, index) => {
const matchesIndent = index < dedentPoints.length - 1 || const matchesIndent = index < dedentPoints.length - 1 || dedentAmount === spaceCount;
dedentAmount === spaceCount; const actualDedentAmount = index < dedentPoints.length - 1 ? dedentAmount : spaceCount;
const actualDedentAmount = index < dedentPoints.length - 1 ? this._tokens.push(
dedentAmount : spaceCount; DedentToken.create(this._cs.position, 0, actualDedentAmount, matchesIndent, this._getComments())
this._tokens.push(DedentToken.create(this._cs.position, 0, actualDedentAmount, );
matchesIndent, this._getComments()));
}); });
} }
} }
@ -589,11 +590,11 @@ export class Tokenizer {
if (this._cs.position > start) { if (this._cs.position > start) {
const value = this._cs.getText().substr(start, this._cs.position - start); const value = this._cs.getText().substr(start, this._cs.position - start);
if (_keywords[value] !== undefined) { if (_keywords[value] !== undefined) {
this._tokens.push(KeywordToken.create(start, this._cs.position - start, this._tokens.push(
_keywords[value], this._getComments())); KeywordToken.create(start, this._cs.position - start, _keywords[value], this._getComments())
);
} else { } else {
this._tokens.push(IdentifierToken.create(start, this._cs.position - start, this._tokens.push(IdentifierToken.create(start, this._cs.position - start, value, this._getComments()));
value, this._getComments()));
} }
return true; return true;
} }
@ -670,7 +671,10 @@ export class Tokenizer {
mightBeFloatingPoint = true; mightBeFloatingPoint = true;
this._cs.moveNext(); this._cs.moveNext();
} }
isDecimalInteger = this._cs.currentChar !== Char.Period && this._cs.currentChar !== Char.e && this._cs.currentChar !== Char.E; isDecimalInteger =
this._cs.currentChar !== Char.Period &&
this._cs.currentChar !== Char.e &&
this._cs.currentChar !== Char.E;
} }
// "0" (["_"] "0")* // "0" (["_"] "0")*
@ -679,7 +683,10 @@ export class Tokenizer {
while (this._cs.currentChar === Char._0 || this._cs.currentChar === Char.Underscore) { while (this._cs.currentChar === Char._0 || this._cs.currentChar === Char.Underscore) {
this._cs.moveNext(); this._cs.moveNext();
} }
isDecimalInteger = this._cs.currentChar !== Char.Period && this._cs.currentChar !== Char.e && this._cs.currentChar !== Char.E; isDecimalInteger =
this._cs.currentChar !== Char.Period &&
this._cs.currentChar !== Char.e &&
this._cs.currentChar !== Char.E;
} }
if (isDecimalInteger) { if (isDecimalInteger) {
@ -692,15 +699,19 @@ export class Tokenizer {
text += String.fromCharCode(this._cs.currentChar); text += String.fromCharCode(this._cs.currentChar);
this._cs.moveNext(); this._cs.moveNext();
} }
this._tokens.push(NumberToken.create(start, text.length, value, true, isImaginary, this._getComments())); this._tokens.push(
NumberToken.create(start, text.length, value, true, isImaginary, this._getComments())
);
return true; return true;
} }
} }
// Floating point. Sign and leading digits were already skipped over. // Floating point. Sign and leading digits were already skipped over.
this._cs.position = start; this._cs.position = start;
if (mightBeFloatingPoint || if (
(this._cs.currentChar === Char.Period && this._cs.nextChar >= Char._0 && this._cs.nextChar <= Char._9)) { mightBeFloatingPoint ||
(this._cs.currentChar === Char.Period && this._cs.nextChar >= Char._0 && this._cs.nextChar <= Char._9)
) {
if (this._skipFloatingPointCandidate()) { if (this._skipFloatingPointCandidate()) {
let text = this._cs.getText().substr(start, this._cs.position - start); let text = this._cs.getText().substr(start, this._cs.position - start);
const value = parseFloat(text); const value = parseFloat(text);
@ -711,8 +722,16 @@ export class Tokenizer {
text += String.fromCharCode(this._cs.currentChar); text += String.fromCharCode(this._cs.currentChar);
this._cs.moveNext(); this._cs.moveNext();
} }
this._tokens.push(NumberToken.create(start, this._cs.position - start, value, this._tokens.push(
false, isImaginary, this._getComments())); NumberToken.create(
start,
this._cs.position - start,
value,
false,
isImaginary,
this._getComments()
)
);
return true; return true;
} }
} }
@ -899,7 +918,10 @@ export class Tokenizer {
} }
if (this._cs.lookAhead(2) === Char.SingleQuote || this._cs.lookAhead(2) === Char.DoubleQuote) { if (this._cs.lookAhead(2) === Char.SingleQuote || this._cs.lookAhead(2) === Char.DoubleQuote) {
const prefix = this._cs.getText().substr(this._cs.position, 2).toLowerCase(); const prefix = this._cs
.getText()
.substr(this._cs.position, 2)
.toLowerCase();
switch (prefix) { switch (prefix) {
case 'rf': case 'rf':
case 'fr': case 'fr':
@ -973,12 +995,20 @@ export class Tokenizer {
const end = this._cs.position; const end = this._cs.position;
this._tokens.push(StringToken.create(start, end - start, stringLiteralInfo.flags, this._tokens.push(
stringLiteralInfo.escapedValue, stringPrefixLength, this._getComments())); StringToken.create(
start,
end - start,
stringLiteralInfo.flags,
stringLiteralInfo.escapedValue,
stringPrefixLength,
this._getComments()
)
);
} }
private _skipToEndOfStringLiteral(flags: StringTokenFlags): StringScannerOutput { private _skipToEndOfStringLiteral(flags: StringTokenFlags): StringScannerOutput {
const quoteChar = (flags & StringTokenFlags.SingleQuote) ? Char.SingleQuote : Char.DoubleQuote; const quoteChar = flags & StringTokenFlags.SingleQuote ? Char.SingleQuote : Char.DoubleQuote;
const isTriplicate = (flags & StringTokenFlags.Triplicate) !== 0; const isTriplicate = (flags & StringTokenFlags.Triplicate) !== 0;
let escapedValue = ''; let escapedValue = '';
@ -1026,9 +1056,12 @@ export class Tokenizer {
} else if (!isTriplicate && this._cs.currentChar === quoteChar) { } else if (!isTriplicate && this._cs.currentChar === quoteChar) {
this._cs.moveNext(); this._cs.moveNext();
break; break;
} else if (isTriplicate && this._cs.currentChar === quoteChar && } else if (
this._cs.nextChar === quoteChar && this._cs.lookAhead(2) === quoteChar) { isTriplicate &&
this._cs.currentChar === quoteChar &&
this._cs.nextChar === quoteChar &&
this._cs.lookAhead(2) === quoteChar
) {
this._cs.advance(3); this._cs.advance(3);
break; break;
} else { } else {

58
server/src/parser/tokenizerTypes.ts
View File

@ -188,9 +188,7 @@ export interface TokenBase extends TextRange {
export interface Token extends TokenBase {} export interface Token extends TokenBase {}
export namespace Token { export namespace Token {
export function create(type: TokenType, start: number, length: number, export function create(type: TokenType, start: number, length: number, comments: Comment[] | undefined) {
comments: Comment[] | undefined) {
const token: Token = { const token: Token = {
start, start,
length, length,
@ -208,9 +206,7 @@ export interface IndentToken extends Token {
} }
export namespace IndentToken { export namespace IndentToken {
export function create(start: number, length: number, indentAmount: number, export function create(start: number, length: number, indentAmount: number, comments: Comment[] | undefined) {
comments: Comment[] | undefined) {
const token: IndentToken = { const token: IndentToken = {
start, start,
length, length,
@ -230,9 +226,13 @@ export interface DedentToken extends Token {
} }
export namespace DedentToken { export namespace DedentToken {
export function create(start: number, length: number, indentAmount: number, export function create(
matchesIndent: boolean, comments: Comment[] | undefined) { start: number,
length: number,
indentAmount: number,
matchesIndent: boolean,
comments: Comment[] | undefined
) {
const token: DedentToken = { const token: DedentToken = {
start, start,
length, length,
@ -252,9 +252,7 @@ export interface NewLineToken extends Token {
} }
export namespace NewLineToken { export namespace NewLineToken {
export function create(start: number, length: number, newLineType: NewLineType, export function create(start: number, length: number, newLineType: NewLineType, comments: Comment[] | undefined) {
comments: Comment[] | undefined) {
const token: NewLineToken = { const token: NewLineToken = {
start, start,
length, length,
@ -273,9 +271,7 @@ export interface KeywordToken extends Token {
} }
export namespace KeywordToken { export namespace KeywordToken {
export function create(start: number, length: number, keywordType: KeywordType, export function create(start: number, length: number, keywordType: KeywordType, comments: Comment[] | undefined) {
comments: Comment[] | undefined) {
const token: KeywordToken = { const token: KeywordToken = {
start, start,
length, length,
@ -305,9 +301,14 @@ export interface StringToken extends Token {
} }
export namespace StringToken { export namespace StringToken {
export function create(start: number, length: number, flags: StringTokenFlags, escapedValue: string, export function create(
prefixLength: number, comments: Comment[] | undefined) { start: number,
length: number,
flags: StringTokenFlags,
escapedValue: string,
prefixLength: number,
comments: Comment[] | undefined
) {
const token: StringToken = { const token: StringToken = {
start, start,
length, length,
@ -315,7 +316,7 @@ export namespace StringToken {
flags, flags,
escapedValue, escapedValue,
prefixLength, prefixLength,
quoteMarkLength: (flags & StringTokenFlags.Triplicate) ? 3 : 1, quoteMarkLength: flags & StringTokenFlags.Triplicate ? 3 : 1,
comments comments
}; };
@ -331,9 +332,14 @@ export interface NumberToken extends Token {
} }
export namespace NumberToken { export namespace NumberToken {
export function create(start: number, length: number, value: number, isInteger: boolean, export function create(
isImaginary: boolean, comments: Comment[] | undefined) { start: number,
length: number,
value: number,
isInteger: boolean,
isImaginary: boolean,
comments: Comment[] | undefined
) {
const token: NumberToken = { const token: NumberToken = {
start, start,
length, length,
@ -354,9 +360,7 @@ export interface OperatorToken extends Token {
} }
export namespace OperatorToken { export namespace OperatorToken {
export function create(start: number, length: number, operatorType: OperatorType, export function create(start: number, length: number, operatorType: OperatorType, comments: Comment[] | undefined) {
comments: Comment[] | undefined) {
const token: OperatorToken = { const token: OperatorToken = {
start, start,
length, length,
@ -375,9 +379,7 @@ export interface IdentifierToken extends Token {
} }
export namespace IdentifierToken { export namespace IdentifierToken {
export function create(start: number, length: number, value: string, export function create(start: number, length: number, value: string, comments: Comment[] | undefined) {
comments: Comment[] | undefined) {
const token: IdentifierToken = { const token: IdentifierToken = {
start, start,
length, length,

2561
server/src/parser/unicode.ts
View File

File diff suppressed because it is too large Load Diff

40
server/src/pyright.ts
View File

@ -168,8 +168,11 @@ function processArgs() {
let errorCount = 0; let errorCount = 0;
if (results.diagnostics.length > 0 && !args.createstub) { if (results.diagnostics.length > 0 && !args.createstub) {
if (args.outputjson) { if (args.outputjson) {
const report = reportDiagnosticsAsJson(results.diagnostics, const report = reportDiagnosticsAsJson(
results.filesInProgram, results.elapsedTime); results.diagnostics,
results.filesInProgram,
results.elapsedTime
);
errorCount += report.errorCount; errorCount += report.errorCount;
} else { } else {
const report = reportDiagnosticsAsText(results.diagnostics); const report = reportDiagnosticsAsText(results.diagnostics);
@ -209,10 +212,7 @@ function processArgs() {
} }
if (!watch) { if (!watch) {
process.exit( process.exit(errorCount > 0 ? ExitStatus.ErrorsReported : ExitStatus.NoErrors);
errorCount > 0 ?
ExitStatus.ErrorsReported :
ExitStatus.NoErrors);
} else { } else {
console.log('Watching for file changes...'); console.log('Watching for file changes...');
} }
@ -230,7 +230,9 @@ function processArgs() {
function printUsage() { function printUsage() {
console.log( console.log(
'Usage: ' + toolName + ' [options] files...\n' + 'Usage: ' +
toolName +
' [options] files...\n' +
' Options:\n' + ' Options:\n' +
' --createstub IMPORT Create type stub file(s) for import\n' + ' --createstub IMPORT Create type stub file(s) for import\n' +
' --dependencies Emit import dependency information\n' + ' --dependencies Emit import dependency information\n' +
@ -256,9 +258,11 @@ function printVersion() {
console.log(`${toolName} ${getVersionString()}`); console.log(`${toolName} ${getVersionString()}`);
} }
function reportDiagnosticsAsJson(fileDiagnostics: FileDiagnostics[], filesInProgram: number, function reportDiagnosticsAsJson(
timeInSec: number): DiagnosticResult { fileDiagnostics: FileDiagnostics[],
filesInProgram: number,
timeInSec: number
): DiagnosticResult {
const report: PyrightJsonResults = { const report: PyrightJsonResults = {
version: getVersionString(), version: getVersionString(),
time: Date.now().toString(), time: Date.now().toString(),
@ -312,19 +316,22 @@ function reportDiagnosticsAsText(fileDiagnostics: FileDiagnostics[]): Diagnostic
fileDiagnostics.forEach(fileDiagnostics => { fileDiagnostics.forEach(fileDiagnostics => {
// Don't report unused code diagnostics. // Don't report unused code diagnostics.
const fileErrorsAndWarnings = fileDiagnostics.diagnostics.filter( const fileErrorsAndWarnings = fileDiagnostics.diagnostics.filter(
diag => diag.category !== DiagnosticCategory.UnusedCode); diag => diag.category !== DiagnosticCategory.UnusedCode
);
if (fileErrorsAndWarnings.length > 0) { if (fileErrorsAndWarnings.length > 0) {
console.log(`${fileDiagnostics.filePath}`); console.log(`${fileDiagnostics.filePath}`);
fileErrorsAndWarnings.forEach(diag => { fileErrorsAndWarnings.forEach(diag => {
let message = ' '; let message = ' ';
if (diag.range) { if (diag.range) {
message += chalk.yellow(`${ diag.range.start.line + 1 }`) + ':' + message +=
chalk.yellow(`${ diag.range.start.character + 1 }`) + ' - '; chalk.yellow(`${diag.range.start.line + 1}`) +
':' +
chalk.yellow(`${diag.range.start.character + 1}`) +
' - ';
} }
message += diag.category === DiagnosticCategory.Error ? message += diag.category === DiagnosticCategory.Error ? chalk.red('error') : chalk.green('warning');
chalk.red('error') : chalk.green('warning');
message += `: ${diag.message}`; message += `: ${diag.message}`;
const rule = diag.getRule(); const rule = diag.getRule();
@ -345,7 +352,8 @@ function reportDiagnosticsAsText(fileDiagnostics: FileDiagnostics[]): Diagnostic
console.log( console.log(
`${errorCount.toString()} ${errorCount === 1 ? 'error' : 'errors'}, ` + `${errorCount.toString()} ${errorCount === 1 ? 'error' : 'errors'}, ` +
`${ warningCount.toString() } ${ warningCount === 1 ? 'warning' : 'warnings' } `); `${warningCount.toString()} ${warningCount === 1 ? 'warning' : 'warnings'} `
);
return { return {
errorCount, errorCount,

182
server/src/tests/checker.test.ts
View File

@ -29,37 +29,157 @@ test('Builtins1', () => {
// `dir(builtins)` // `dir(builtins)`
// Remove True, False, None, _, __build_class__, __debug__, __doc__ // Remove True, False, None, _, __build_class__, __debug__, __doc__
const expectedBuiltinsSymbols = [ const expectedBuiltinsSymbols = [
'ArithmeticError', 'AssertionError', 'AttributeError', 'BaseException', 'ArithmeticError',
'BlockingIOError', 'BrokenPipeError', 'BufferError', 'BytesWarning', 'AssertionError',
'ChildProcessError', 'ConnectionAbortedError', 'ConnectionError', 'AttributeError',
'ConnectionRefusedError', 'ConnectionResetError', 'DeprecationWarning', 'BaseException',
'EOFError', 'Ellipsis', 'EnvironmentError', 'Exception', 'BlockingIOError',
'FileExistsError', 'FileNotFoundError', 'FloatingPointError', 'BrokenPipeError',
'FutureWarning', 'GeneratorExit', 'IOError', 'ImportError', 'BufferError',
'ImportWarning', 'IndentationError', 'IndexError', 'InterruptedError', 'BytesWarning',
'IsADirectoryError', 'KeyError', 'KeyboardInterrupt', 'LookupError', 'ModuleNotFoundError', 'ChildProcessError',
'MemoryError', 'NameError', 'NotADirectoryError', 'NotImplemented', 'ConnectionAbortedError',
'NotImplementedError', 'OSError', 'OverflowError', 'PendingDeprecationWarning', 'ConnectionError',
'PermissionError', 'ProcessLookupError', 'RecursionError', 'ReferenceError', 'ConnectionRefusedError',
'ResourceWarning', 'RuntimeError', 'RuntimeWarning', 'StopAsyncIteration', 'ConnectionResetError',
'StopIteration', 'SyntaxError', 'SyntaxWarning', 'SystemError', 'SystemExit', 'DeprecationWarning',
'TabError', 'TimeoutError', 'TypeError', 'UnboundLocalError', 'EOFError',
'UnicodeDecodeError', 'UnicodeEncodeError', 'UnicodeError', 'UnicodeTranslateError', 'Ellipsis',
'UnicodeWarning', 'UserWarning', 'ValueError', 'Warning', 'WindowsError', 'EnvironmentError',
'Exception',
'FileExistsError',
'FileNotFoundError',
'FloatingPointError',
'FutureWarning',
'GeneratorExit',
'IOError',
'ImportError',
'ImportWarning',
'IndentationError',
'IndexError',
'InterruptedError',
'IsADirectoryError',
'KeyError',
'KeyboardInterrupt',
'LookupError',
'ModuleNotFoundError',
'MemoryError',
'NameError',
'NotADirectoryError',
'NotImplemented',
'NotImplementedError',
'OSError',
'OverflowError',
'PendingDeprecationWarning',
'PermissionError',
'ProcessLookupError',
'RecursionError',
'ReferenceError',
'ResourceWarning',
'RuntimeError',
'RuntimeWarning',
'StopAsyncIteration',
'StopIteration',
'SyntaxError',
'SyntaxWarning',
'SystemError',
'SystemExit',
'TabError',
'TimeoutError',
'TypeError',
'UnboundLocalError',
'UnicodeDecodeError',
'UnicodeEncodeError',
'UnicodeError',
'UnicodeTranslateError',
'UnicodeWarning',
'UserWarning',
'ValueError',
'Warning',
'WindowsError',
'ZeroDivisionError', 'ZeroDivisionError',
'__import__', '__loader__', '__name__', '__import__',
'__package__', '__spec__', 'abs', 'all', 'any', 'ascii', 'bin', 'bool', 'breakpoint', '__loader__',
'bytearray', 'bytes', 'callable', 'chr', 'classmethod', 'compile', 'complex', '__name__',
'copyright', 'credits', 'delattr', 'dict', 'dir', 'divmod', 'enumerate', 'eval', '__package__',
'exec', 'exit', 'filter', 'float', 'format', 'frozenset', 'getattr', 'globals', '__spec__',
'hasattr', 'hash', 'help', 'hex', 'id', 'input', 'int', 'isinstance', 'abs',
'issubclass', 'iter', 'len', 'license', 'list', 'locals', 'map', 'max', 'all',
'memoryview', 'min', 'next', 'object', 'oct', 'open', 'ord', 'pow', 'print', 'any',
'property', 'quit', 'range', 'repr', 'reversed', 'round', 'set', 'setattr', 'ascii',
'slice', 'sorted', 'staticmethod', 'str', 'sum', 'super', 'tuple', 'type', 'bin',
'vars', 'zip', 'bool',
'breakpoint',
'bytearray',
'bytes',
'callable',
'chr',
'classmethod',
'compile',
'complex',
'copyright',
'credits',
'delattr',
'dict',
'dir',
'divmod',
'enumerate',
'eval',
'exec',
'exit',
'filter',
'float',
'format',
'frozenset',
'getattr',
'globals',
'hasattr',
'hash',
'help',
'hex',
'id',
'input',
'int',
'isinstance',
'issubclass',
'iter',
'len',
'license',
'list',
'locals',
'map',
'max',
'memoryview',
'min',
'next',
'object',
'oct',
'open',
'ord',
'pow',
'print',
'property',
'quit',
'range',
'repr',
'reversed',
'round',
'set',
'setattr',
'slice',
'sorted',
'staticmethod',
'str',
'sum',
'super',
'tuple',
'type',
'vars',
'zip',
// These really shouldn't be exposed but are defined by builtins.pyi currently. // These really shouldn't be exposed but are defined by builtins.pyi currently.
'function', 'ellipsis']; 'function',
'ellipsis'
];
const moduleScope = AnalyzerNodeInfo.getScope(analysisResults[0].parseResults!.parseTree)!; const moduleScope = AnalyzerNodeInfo.getScope(analysisResults[0].parseResults!.parseTree)!;
assert.notEqual(moduleScope, undefined); assert.notEqual(moduleScope, undefined);
@ -94,9 +214,7 @@ test('Builtins1', () => {
} }
}); });
function validateResults(results: TestUtils.FileAnalysisResult[], function validateResults(results: TestUtils.FileAnalysisResult[], errorCount: number, warningCount = 0) {
errorCount: number, warningCount = 0) {
assert.equal(results.length, 1); assert.equal(results.length, 1);
assert.equal(results[0].errors.length, errorCount); assert.equal(results[0].errors.length, errorCount);
assert.equal(results[0].warnings.length, warningCount); assert.equal(results[0].warnings.length, warningCount);

31
server/src/tests/collectionUtils.test.ts
View File

@ -36,29 +36,44 @@ test('UtilsAppendUndefinedValue', () => {
test('UtilsFindEmpty', () => { test('UtilsFindEmpty', () => {
const data: number[] = []; const data: number[] = [];
assert.equal(utils.find(data, e => true), undefined); assert.equal(
utils.find(data, e => true),
undefined
);
}); });
test('UtilsFindNoMatch', () => { test('UtilsFindNoMatch', () => {
const data = [1]; const data = [1];
assert.equal(utils.find(data, e => false), undefined); assert.equal(
utils.find(data, e => false),
undefined
);
}); });
test('UtilsFindMatchSimple', () => { test('UtilsFindMatchSimple', () => {
const data = [1]; const data = [1];
assert.equal(utils.find(data, e => e === 1), 1); assert.equal(
utils.find(data, e => e === 1),
1
);
}); });
test('UtilsFindMatch', () => { test('UtilsFindMatch', () => {
const data = [new D(1, 'Hello')]; const data = [new D(1, 'Hello')];
assert.equal(utils.find(data, e => e.value === 1), data[0]); assert.equal(
utils.find(data, e => e.value === 1),
data[0]
);
}); });
test('UtilsFindMatchCovariant', () => { test('UtilsFindMatchCovariant', () => {
const item1 = new D(1, 'Hello'); const item1 = new D(1, 'Hello');
const item2 = new D(2, 'Hello2'); const item2 = new D(2, 'Hello2');
const data: B[] = [new B(0), item1, item2, new B(3)]; const data: B[] = [new B(0), item1, item2, new B(3)];
assert.equal(utils.find(data, (e: D) => e.value === 2), item2); assert.equal(
utils.find(data, (e: D) => e.value === 2),
item2
);
}); });
test('UtilsStableSort', () => { test('UtilsStableSort', () => {
@ -126,7 +141,11 @@ test('cloneAndSort', () => {
}); });
test('flatten', () => { test('flatten', () => {
const data: number[][] = [[1, 2], [3, 4], [5, 6]]; const data: number[][] = [
[1, 2],
[3, 4],
[5, 6]
];
assert.deepEqual(utils.flatten(data), [1, 2, 3, 4, 5, 6]); assert.deepEqual(utils.flatten(data), [1, 2, 3, 4, 5, 6]);
}); });

29
server/src/tests/config.test.ts
View File

@ -17,7 +17,7 @@ import { combinePaths, normalizePath,normalizeSlashes } from '../common/pathUtil
import { createFromRealFileSystem } from '../common/vfs'; import { createFromRealFileSystem } from '../common/vfs';
test('FindFilesWithConfigFile', () => { test('FindFilesWithConfigFile', () => {
const cwd = normalizePath(combinePaths(process.cwd(), "../server")) const cwd = normalizePath(combinePaths(process.cwd(), '../server'));
const service = new AnalyzerService('<default>', createFromRealFileSystem(), new NullConsole()); const service = new AnalyzerService('<default>', createFromRealFileSystem(), new NullConsole());
const commandLineOptions = new CommandLineOptions(cwd, true); const commandLineOptions = new CommandLineOptions(cwd, true);
commandLineOptions.configFilePath = 'src/tests/samples/project1'; commandLineOptions.configFilePath = 'src/tests/samples/project1';
@ -27,8 +27,10 @@ test('FindFilesWithConfigFile', () => {
// The config file specifies a single file spec (a directory). // The config file specifies a single file spec (a directory).
assert.equal(configOptions.include.length, 1, `failed creating options from ${cwd}`); assert.equal(configOptions.include.length, 1, `failed creating options from ${cwd}`);
assert.equal(normalizeSlashes(configOptions.projectRoot), assert.equal(
normalizeSlashes(combinePaths(cwd, commandLineOptions.configFilePath))); normalizeSlashes(configOptions.projectRoot),
normalizeSlashes(combinePaths(cwd, commandLineOptions.configFilePath))
);
const fileList = service.test_getFileNamesFromFileSpecs(); const fileList = service.test_getFileNamesFromFileSpecs();
@ -39,7 +41,7 @@ test('FindFilesWithConfigFile', () => {
}); });
test('FileSpecNotAnArray', () => { test('FileSpecNotAnArray', () => {
const cwd = normalizePath(combinePaths(process.cwd(), "../server")) const cwd = normalizePath(combinePaths(process.cwd(), '../server'));
const nullConsole = new NullConsole(); const nullConsole = new NullConsole();
const service = new AnalyzerService('<default>', createFromRealFileSystem(nullConsole), nullConsole); const service = new AnalyzerService('<default>', createFromRealFileSystem(nullConsole), nullConsole);
const commandLineOptions = new CommandLineOptions(cwd, false); const commandLineOptions = new CommandLineOptions(cwd, false);
@ -53,7 +55,7 @@ test('FileSpecNotAnArray', () => {
}); });
test('FileSpecNotAString', () => { test('FileSpecNotAString', () => {
const cwd = normalizePath(combinePaths(process.cwd(), "../server")) const cwd = normalizePath(combinePaths(process.cwd(), '../server'));
const nullConsole = new NullConsole(); const nullConsole = new NullConsole();
const service = new AnalyzerService('<default>', createFromRealFileSystem(nullConsole), nullConsole); const service = new AnalyzerService('<default>', createFromRealFileSystem(nullConsole), nullConsole);
const commandLineOptions = new CommandLineOptions(cwd, false); const commandLineOptions = new CommandLineOptions(cwd, false);
@ -67,7 +69,7 @@ test('FileSpecNotAString', () => {
}); });
test('SomeFileSpecsAreInvalid', () => { test('SomeFileSpecsAreInvalid', () => {
const cwd = normalizePath(combinePaths(process.cwd(), "../server")) const cwd = normalizePath(combinePaths(process.cwd(), '../server'));
const nullConsole = new NullConsole(); const nullConsole = new NullConsole();
const service = new AnalyzerService('<default>', createFromRealFileSystem(nullConsole), nullConsole); const service = new AnalyzerService('<default>', createFromRealFileSystem(nullConsole), nullConsole);
const commandLineOptions = new CommandLineOptions(cwd, false); const commandLineOptions = new CommandLineOptions(cwd, false);
@ -80,8 +82,10 @@ test('SomeFileSpecsAreInvalid', () => {
// and one in the exclude array. // and one in the exclude array.
assert.equal(configOptions.include.length, 4, `failed creating options from ${cwd}`); assert.equal(configOptions.include.length, 4, `failed creating options from ${cwd}`);
assert.equal(configOptions.exclude.length, 1); assert.equal(configOptions.exclude.length, 1);
assert.equal(normalizeSlashes(configOptions.projectRoot), assert.equal(
normalizeSlashes(combinePaths(cwd, commandLineOptions.configFilePath))); normalizeSlashes(configOptions.projectRoot),
normalizeSlashes(combinePaths(cwd, commandLineOptions.configFilePath))
);
const fileList = service.test_getFileNamesFromFileSpecs(); const fileList = service.test_getFileNamesFromFileSpecs();
@ -90,7 +94,7 @@ test('SomeFileSpecsAreInvalid', () => {
}); });
test('ConfigBadJson', () => { test('ConfigBadJson', () => {
const cwd = normalizePath(combinePaths(process.cwd(), "../server")) const cwd = normalizePath(combinePaths(process.cwd(), '../server'));
const nullConsole = new NullConsole(); const nullConsole = new NullConsole();
const service = new AnalyzerService('<default>', createFromRealFileSystem(nullConsole), nullConsole); const service = new AnalyzerService('<default>', createFromRealFileSystem(nullConsole), nullConsole);
const commandLineOptions = new CommandLineOptions(cwd, false); const commandLineOptions = new CommandLineOptions(cwd, false);
@ -104,7 +108,7 @@ test('ConfigBadJson', () => {
}); });
test('FindExecEnv1', () => { test('FindExecEnv1', () => {
const cwd = normalizePath(combinePaths(process.cwd(), "../server")) const cwd = normalizePath(combinePaths(process.cwd(), '../server'));
const configOptions = new ConfigOptions(cwd); const configOptions = new ConfigOptions(cwd);
// Build a config option with three execution environments. // Build a config option with three execution environments.
@ -122,12 +126,11 @@ test('FindExecEnv1', () => {
// a default environment with the root equal to that of the config. // a default environment with the root equal to that of the config.
const file4 = '/nothing/bar.py'; const file4 = '/nothing/bar.py';
const defaultExecEnv = configOptions.findExecEnvironment(file4); const defaultExecEnv = configOptions.findExecEnvironment(file4);
assert.equal(normalizeSlashes(defaultExecEnv.root), assert.equal(normalizeSlashes(defaultExecEnv.root), normalizeSlashes(configOptions.projectRoot));
normalizeSlashes(configOptions.projectRoot));
}); });
test('PythonPlatform', () => { test('PythonPlatform', () => {
const cwd = normalizePath(combinePaths(process.cwd(), "../server")) const cwd = normalizePath(combinePaths(process.cwd(), '../server'));
const nullConsole = new NullConsole(); const nullConsole = new NullConsole();
const configOptions = new ConfigOptions(cwd); const configOptions = new ConfigOptions(cwd);

46
server/src/tests/debug.test.ts
View File

@ -9,14 +9,19 @@ import * as assert from 'assert';
import * as debug from '../common/debug'; import * as debug from '../common/debug';
test('DebugAssertTrue', () => { test('DebugAssertTrue', () => {
assert.doesNotThrow(() => { debug.assert(true, 'doesn\'t throw'); }); assert.doesNotThrow(() => {
debug.assert(true, "doesn't throw");
});
}); });
test('DebugAssertFalse', () => { test('DebugAssertFalse', () => {
assert.throws( assert.throws(
() => { debug.assert(false, 'should throw'); }, () => {
debug.assert(false, 'should throw');
},
(err: any) => err instanceof Error, (err: any) => err instanceof Error,
'unexpected'); 'unexpected'
);
}); });
test('DebugAssertDetailInfo', () => { test('DebugAssertDetailInfo', () => {
@ -24,17 +29,23 @@ test('DebugAssertDetailInfo', () => {
// assert raised // assert raised
const detailInfo = 'Detail Info'; const detailInfo = 'Detail Info';
assert.throws( assert.throws(
() => { debug.assert(false, 'should throw', () => detailInfo); }, () => {
debug.assert(false, 'should throw', () => detailInfo);
},
(err: any) => err instanceof Error && err.message.includes(detailInfo), (err: any) => err instanceof Error && err.message.includes(detailInfo),
'unexpected'); 'unexpected'
);
}); });
test('DebugAssertStackTrace', () => { test('DebugAssertStackTrace', () => {
// let assert to control what callstack to put in exception stack // let assert to control what callstack to put in exception stack
assert.throws( assert.throws(
() => { debug.assert(false, 'should throw', undefined, assert.throws); }, () => {
debug.assert(false, 'should throw', undefined, assert.throws);
},
(err: any) => err instanceof Error && !err.message.includes('assert.throws'), (err: any) => err instanceof Error && !err.message.includes('assert.throws'),
'unexpected'); 'unexpected'
);
}); });
test('DebugAssertUndefined', () => { test('DebugAssertUndefined', () => {
@ -42,7 +53,8 @@ test('DebugAssertUndefined', () => {
assert.throws( assert.throws(
() => debug.assertDefined(unused), () => debug.assertDefined(unused),
(err: any) => err instanceof Error, (err: any) => err instanceof Error,
'unexpected'); 'unexpected'
);
}); });
test('DebugAssertDefined', () => { test('DebugAssertDefined', () => {
@ -56,7 +68,8 @@ test('DebugAssertEachUndefined', () => {
assert.throws( assert.throws(
() => debug.assertEachDefined(unused), () => debug.assertEachDefined(unused),
(err: any) => err instanceof Error, (err: any) => err instanceof Error,
'unexpected'); 'unexpected'
);
}); });
test('DebugAssertEachDefined', () => { test('DebugAssertEachDefined', () => {
@ -65,7 +78,11 @@ test('DebugAssertEachDefined', () => {
}); });
test('DebugAssertNever', () => { test('DebugAssertNever', () => {
const enum MyEnum { A, B, C } const enum MyEnum {
A,
B,
C
}
const unused = 5 as MyEnum; const unused = 5 as MyEnum;
// prevent one from adding new values and forget to add // prevent one from adding new values and forget to add
@ -82,7 +99,8 @@ test('DebugAssertNever', () => {
} }
}, },
(err: any) => err instanceof Error, (err: any) => err instanceof Error,
'unexpected'); 'unexpected'
);
}); });
test('DebugGetFunctionName', () => { test('DebugGetFunctionName', () => {
@ -93,6 +111,10 @@ test('DebugGetFunctionName', () => {
test('DebugFormatEnum', () => { test('DebugFormatEnum', () => {
// helper method to add better message in exception around enum // helper method to add better message in exception around enum
// const enum require --preserveConstEnums flag to work properly // const enum require --preserveConstEnums flag to work properly
enum MyEnum { A, B, C } enum MyEnum {
A,
B,
C
}
assert(debug.formatEnum(MyEnum.A, MyEnum, false) === 'A'); assert(debug.formatEnum(MyEnum.A, MyEnum, false) === 'A');
}); });

20
server/src/tests/filesystem.test.ts
View File

@ -24,7 +24,9 @@ test('Folders', () => {
const fs = new vfs.FileSystem(/*ignoreCase*/ true, { cwd }); const fs = new vfs.FileSystem(/*ignoreCase*/ true, { cwd });
// no such dir exist // no such dir exist
assert.throws(() => { fs.chdir('a'); }); assert.throws(() => {
fs.chdir('a');
});
fs.mkdirSync('a'); fs.mkdirSync('a');
fs.chdir('a'); fs.chdir('a');
@ -34,7 +36,9 @@ test('Folders', () => {
fs.rmdirSync('a'); fs.rmdirSync('a');
// no such dir exist // no such dir exist
assert.throws(() => { fs.chdir('a'); }); assert.throws(() => {
fs.chdir('a');
});
}); });
test('Files', () => { test('Files', () => {
@ -145,8 +149,10 @@ test('createFromFileSystem1', () => {
const content = '# test'; const content = '# test';
// file system will map physical file system to virtual one // file system will map physical file system to virtual one
const fs = factory.createFromFileSystem(host.HOST, false, const fs = factory.createFromFileSystem(host.HOST, false, {
{ documents: [new factory.TextDocument(filepath, content)], cwd: factory.srcFolder }); documents: [new factory.TextDocument(filepath, content)],
cwd: factory.srcFolder
});
// check existing typeshed folder on virtual path inherited from base snapshot from physical file system // check existing typeshed folder on virtual path inherited from base snapshot from physical file system
const entries = fs.readdirSync(factory.typeshedFolder); const entries = fs.readdirSync(factory.typeshedFolder);
@ -165,7 +171,8 @@ test('createFromFileSystem2', () => {
test('createFromFileSystemWithCustomTypeshedPath', () => { test('createFromFileSystemWithCustomTypeshedPath', () => {
const invalidpath = normalizeSlashes(combinePaths(host.HOST.getWorkspaceRoot(), '../docs')); const invalidpath = normalizeSlashes(combinePaths(host.HOST.getWorkspaceRoot(), '../docs'));
const fs = factory.createFromFileSystem(host.HOST, /* ignoreCase */ false, { const fs = factory.createFromFileSystem(host.HOST, /* ignoreCase */ false, {
cwd: factory.srcFolder, meta: { [factory.typeshedFolder]: invalidpath } cwd: factory.srcFolder,
meta: { [factory.typeshedFolder]: invalidpath }
}); });
const entries = fs.readdirSync(factory.typeshedFolder); const entries = fs.readdirSync(factory.typeshedFolder);
@ -174,7 +181,8 @@ test('createFromFileSystemWithCustomTypeshedPath', () => {
test('createFromFileSystemWithMetadata', () => { test('createFromFileSystemWithMetadata', () => {
const fs = factory.createFromFileSystem(host.HOST, /* ignoreCase */ false, { const fs = factory.createFromFileSystem(host.HOST, /* ignoreCase */ false, {
cwd: factory.srcFolder, meta: { 'unused': 'unused' } cwd: factory.srcFolder,
meta: { unused: 'unused' }
}); });
assert(fs.existsSync(factory.srcFolder)); assert(fs.existsSync(factory.srcFolder));

29
server/src/tests/fourSlashParser.test.ts
View File

@ -30,7 +30,10 @@ test('GlobalOptions', () => {
pass`; pass`;
const data = parseTestData('.', code, 'test.py'); const data = parseTestData('.', code, 'test.py');
assertOptions(data.globalOptions, [['libpath', '../dist/lib'], ['pythonversion', '3.7']]); assertOptions(data.globalOptions, [
['libpath', '../dist/lib'],
['pythonversion', '3.7']
]);
assert.equal(data.files.length, 1); assert.equal(data.files.length, 1);
assert.equal(data.files[0].fileName, 'test.py'); assert.equal(data.files[0].fileName, 'test.py');
@ -69,7 +72,10 @@ test('Extra file options', () => {
assert.equal(data.files[0].fileName, normalizeSlashes('./file1.py')); assert.equal(data.files[0].fileName, normalizeSlashes('./file1.py'));
assertOptions(data.globalOptions, []); assertOptions(data.globalOptions, []);
assertOptions(data.files[0].fileOptions, [['filename', 'file1.py'], ['library', 'false']]); assertOptions(data.files[0].fileOptions, [
['filename', 'file1.py'],
['library', 'false']
]);
}); });
test('Library options', () => { test('Library options', () => {
@ -215,8 +221,10 @@ test('Multiple Files', () => {
assert.equal(data.files.length, 3); assert.equal(data.files.length, 3);
assert.equal(data.files.filter(f => f.fileName === normalizeSlashes('./src/A.py'))[0].content, getContent('A')); assert.equal(data.files.filter(f => f.fileName === normalizeSlashes('./src/A.py'))[0].content, getContent('A'));
assert.equal(data.files.filter(f => f.fileName === assert.equal(
normalizeSlashes(combinePaths(factory.libFolder, 'src/B.py')))[0].content, getContent('B')); data.files.filter(f => f.fileName === normalizeSlashes(combinePaths(factory.libFolder, 'src/B.py')))[0].content,
getContent('B')
);
assert.equal(data.files.filter(f => f.fileName === normalizeSlashes('./src/C.py'))[0].content, getContent('C')); assert.equal(data.files.filter(f => f.fileName === normalizeSlashes('./src/C.py'))[0].content, getContent('C'));
}); });
@ -290,10 +298,14 @@ test('fourSlashWithFileSystem', () => {
`; `;
const data = parseTestData('.', code, 'unused'); const data = parseTestData('.', code, 'unused');
const documents = data.files.map(f => new factory.TextDocument(f.fileName, f.content, const documents = data.files.map(
new Map<string, string>(Object.entries(f.fileOptions)))); f => new factory.TextDocument(f.fileName, f.content, new Map<string, string>(Object.entries(f.fileOptions)))
);
const fs = factory.createFromFileSystem(host.HOST, /* ignoreCase */ false, { documents, cwd: normalizeSlashes('/') }); const fs = factory.createFromFileSystem(host.HOST, /* ignoreCase */ false, {
documents,
cwd: normalizeSlashes('/')
});
for (const file of data.files) { for (const file of data.files) {
assert.equal(fs.readFileSync(file.fileName, 'utf8'), getContent(getBaseFileName(file.fileName, '.py', false))); assert.equal(fs.readFileSync(file.fileName, 'utf8'), getContent(getBaseFileName(file.fileName, '.py', false)));
@ -309,5 +321,6 @@ function assertOptions(actual: CompilerSettings, expected: [string, string][], m
assert.deepEqual( assert.deepEqual(
Object.entries(actual).sort((x, y) => compareStringsCaseSensitive(x[0], y[0])), Object.entries(actual).sort((x, y) => compareStringsCaseSensitive(x[0], y[0])),
expected, expected,
message); message
);
} }

17
server/src/tests/fourslash/dataclass4.fourslash.ts
View File

@ -50,9 +50,16 @@
//// ddd: InitVar[int] = 3 //// ddd: InitVar[int] = 3
helper.verifyDiagnostics({ helper.verifyDiagnostics({
"marker1": { category: "error", message: "No parameter named 'ddd'" }, marker1: { category: 'error', message: "No parameter named 'ddd'" },
"marker2": { category: "error", message: "Argument of type 'Literal['hello']' cannot be assigned to parameter 'bbb' of type 'int'\n 'str' is incompatible with 'int'" }, marker2: {
"marker3": { category: "error", message: "Argument missing for parameter 'ccc'" }, category: 'error',
"marker4": { category: "error", message: "Expected 3 positional arguments" }, message:
"marker5": { category: "error", message: "Data fields without default value cannot appear after data fields with default values" }, "Argument of type 'Literal['hello']' cannot be assigned to parameter 'bbb' of type 'int'\n 'str' is incompatible with 'int'"
},
marker3: { category: 'error', message: "Argument missing for parameter 'ccc'" },
marker4: { category: 'error', message: 'Expected 3 positional arguments' },
marker5: {
category: 'error',
message: 'Data fields without default value cannot appear after data fields with default values'
}
}); });

4
server/src/tests/fourslash/dataclass5.fourslash.ts
View File

@ -48,6 +48,6 @@
//// d = [|/*marker1*/[|/*marker2*/D(3|]|]) //// d = [|/*marker1*/[|/*marker2*/D(3|]|])
helper.verifyDiagnostics({ helper.verifyDiagnostics({
"marker1": { category: "error", message: "Expected no arguments to 'D' constructor" }, marker1: { category: 'error', message: "Expected no arguments to 'D' constructor" },
"marker2": { category: "error", message: "'D(3)' has type 'Type[D]' and is not callable" }, marker2: { category: 'error', message: "'D(3)' has type 'Type[D]' and is not callable" }
}); });

4
server/src/tests/fourslash/dataclass6.fourslash.ts
View File

@ -29,6 +29,6 @@
//// ////
helper.verifyDiagnostics({ helper.verifyDiagnostics({
"marker1": { category: "error", message: "Expected 1 positional argument" }, marker1: { category: 'error', message: 'Expected 1 positional argument' },
"marker2": { category: "error", message: "Argument missing for parameter 'y'" }, marker2: { category: 'error', message: "Argument missing for parameter 'y'" }
}); });

Some files were not shown because too many files have changed in this diff Show More