microsoft/pyright
1
1
Fork 0
mirror of https://github.com/microsoft/pyright.git synced 2024-07-07 04:26:31 +03:00
Code Issues Projects Releases Wiki Activity

Mass reformat ts/js/md with ESLint quick fixes and prettier (#113)

Browse Source
This commit is contained in:
Jake Bailey 2020-02-24 16:52:23 -08:00 committed by GitHub
parent e17977c928
commit 8d2cca7289
143 changed files with 11201 additions and 6061 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

5
.eslintignore
View File

@ -1 +1,4 @@
**/tests/fourslash/** **/dist/**
**/out/**
**/node_modules/**
**/tests/fourslash/**

1
.eslintrc.json
View File

@ -1,6 +1,7 @@
{ {
"extends": [ "extends": [
"eslint:recommended", "eslint:recommended",
"prettier",
"plugin:@typescript-eslint/recommended" "plugin:@typescript-eslint/recommended"
], ],
"env": { "env": {

6
.prettierignore Normal file
View File

@ -0,0 +1,6 @@
*.json
**/dist/**
**/out/**
**/client/server/**
**/typeshed-fallback/**
**/.github/**

6
.prettierrc Normal file
View File

@ -0,0 +1,6 @@
{
"singleQuote": true,
"tabWidth": 4,
"useTabs": false,
"printWidth": 120
}

6
.vscode/extensions.json vendored Normal file
View File

@ -0,0 +1,6 @@
{
"recommendations": [
"dbaeumer.vscode-eslint",
"esbenp.prettier-vscode"
]
}

17
.vscode/settings.json vendored Normal file
View File

@ -0,0 +1,17 @@
{
"[typescript]": {
"editor.defaultFormatter": "esbenp.prettier-vscode",
"editor.formatOnSave": true
},
"[javascript]": {
"editor.defaultFormatter": "esbenp.prettier-vscode",
"editor.formatOnSave": true
},
"[markdown]": {
"editor.defaultFormatter": "esbenp.prettier-vscode",
"editor.formatOnSave": true
},
"editor.codeActionsOnSave": {
"source.fixAll.eslint": true
}
}

81
README.md
View File

@ -3,54 +3,64 @@
# Static type checker for Python # Static type checker for Python
### Speed ### Speed
Pyright is a fast type checker meant for large Python source bases. It can run in a “watch” mode and performs fast incremental updates when files are modified.
Pyright is a fast type checker meant for large Python source bases. It can run in a "watch" mode and performs fast incremental updates when files are modified.
### Configurability ### Configurability
Pyright supports [configuration files](/docs/configuration.md) that provide granular control over settings. Different “execution environments” can be associated with subdirectories within a source base. Each environment can specify different module search paths, python language versions, and platform targets.
Pyright supports [configuration files](/docs/configuration.md) that provide granular control over settings. Different "execution environments" can be associated with subdirectories within a source base. Each environment can specify different module search paths, python language versions, and platform targets.
### Type Checking Features ### Type Checking Features
* [PEP 484](https://www.python.org/dev/peps/pep-0484/) type hints including generics
* [PEP 526](https://www.python.org/dev/peps/pep-0526/) syntax for variable annotations - [PEP 484](https://www.python.org/dev/peps/pep-0484/) type hints including generics
* [PEP 544](https://www.python.org/dev/peps/pep-0544/) structural subtyping - [PEP 526](https://www.python.org/dev/peps/pep-0526/) syntax for variable annotations
* [PEP 589](https://www.python.org/dev/peps/pep-0589/) typed dictionaries - [PEP 544](https://www.python.org/dev/peps/pep-0544/) structural subtyping
* Type inference for function return values, instance variables, class variables, and globals - [PEP 589](https://www.python.org/dev/peps/pep-0589/) typed dictionaries
* Smart type constraints that understand conditional code flow constructs like if/else statements - Type inference for function return values, instance variables, class variables, and globals
- Smart type constraints that understand conditional code flow constructs like if/else statements
### VS Code Integration ### VS Code Integration
Pyright ships as both a command-line tool and a VS Code extension that provides many powerful features that help improve programming efficiency. Pyright ships as both a command-line tool and a VS Code extension that provides many powerful features that help improve programming efficiency.
### VS Code Language Features ### VS Code Language Features
The VS Code extension supports many time-saving language features including: The VS Code extension supports many time-saving language features including:
* Intelligent type completion of keywords, symbols, and import names appears when editing - Intelligent type completion of keywords, symbols, and import names appears when editing
* Import statements are automatically inserted when necessary for type completions - Import statements are automatically inserted when necessary for type completions
* Signature completion tips help when filling in arguments for a call - Signature completion tips help when filling in arguments for a call
* Hover over symbols to provide type information and doc strings - Hover over symbols to provide type information and doc strings
* Find Definitions to quickly go to the location of a symbol’s definition - Find Definitions to quickly go to the location of a symbol's definition
* Find References to find all references to a symbol within a code base - Find References to find all references to a symbol within a code base
* Rename Symbol to rename all references to a symbol within a code base - Rename Symbol to rename all references to a symbol within a code base
* Find Symbols within the current document or within the entire workspace - Find Symbols within the current document or within the entire workspace
* Organize Imports command for automatically ordering imports according to PEP8 rules - Organize Imports command for automatically ordering imports according to PEP8 rules
* Type stub generation for third-party libraries - Type stub generation for third-party libraries
### Built-in Type Stubs ### Built-in Type Stubs
Pyright includes a recent copy of the stdlib type stubs from [Typeshed](https://github.com/python/typeshed). It can be configured to use another (perhaps more recent or modified) copy of the Typeshed type stubs. Of course, it also works with custom type stub files that are part of your project. Pyright includes a recent copy of the stdlib type stubs from [Typeshed](https://github.com/python/typeshed). It can be configured to use another (perhaps more recent or modified) copy of the Typeshed type stubs. Of course, it also works with custom type stub files that are part of your project.
### Command-line Tool or Visual Studio Code Extension ### Command-line Tool or Visual Studio Code Extension
Pyright includes both a [command-line tool](/docs/command-line.md) and an [extension for Visual Studio Code](https://marketplace.visualstudio.com/items?itemName=ms-pyright.pyright) that implements the [Language Server Protocol](https://microsoft.github.io/language-server-protocol/). Pyright includes both a [command-line tool](/docs/command-line.md) and an [extension for Visual Studio Code](https://marketplace.visualstudio.com/items?itemName=ms-pyright.pyright) that implements the [Language Server Protocol](https://microsoft.github.io/language-server-protocol/).
For rich Python editing and debugging capabilities with Visual Studio Code, be sure to also install the official [Microsoft Python extension for Visual Studio Code](https://marketplace.visualstudio.com/itemdetails?itemName=ms-python.python) as Pyright only provides syntax and type checking. For rich Python editing and debugging capabilities with Visual Studio Code, be sure to also install the official [Microsoft Python extension for Visual Studio Code](https://marketplace.visualstudio.com/itemdetails?itemName=ms-python.python) as Pyright only provides syntax and type checking.
## Installation ## Installation
### VS Code Extension ### VS Code Extension
You can install the latest-published version of the Pyright VS Code extension directly from VS Code. Simply open the extensions panel and search for `pyright`. You can install the latest-published version of the Pyright VS Code extension directly from VS Code. Simply open the extensions panel and search for `pyright`.
### Vim ### Vim
For vim/neovim users, you can install [coc-pyright](https://github.com/fannheyward/coc-pyright), Pyright extension for coc.nvim. For vim/neovim users, you can install [coc-pyright](https://github.com/fannheyward/coc-pyright), Pyright extension for coc.nvim.
### Command-line ### Command-line
The latest version of the command-line tool can be installed with npm, which is part of node. If you don't have a recent version of node on your system, install that first from [nodejs.org](nodejs.org).
The latest version of the command-line tool can be installed with npm, which is part of node. If you don't have a recent version of node on your system, install that first from [nodejs.org](nodejs.org).
To install pyright globally: To install pyright globally:
`npm install -g pyright` `npm install -g pyright`
@ -64,42 +74,41 @@ Once installed, you can run the tool from the command line as follows:
To update to the latest version: To update to the latest version:
`sudo npm update -g pyright` `sudo npm update -g pyright`
## Using Pyright with VS Code Python Extension ## Using Pyright with VS Code Python Extension
Pyright provides some features that overlap with functionality provided by the standard VS Code Python extension: “hover”, type completion, definitions, references, rename symbols, etc. You may see duplicate results if Pyright is installed alongside the Python extension. There is currently no way to disable this functionality in the Python extension. If you want to disable these features in Pyright, there is a setting to do so: `pyright.disableLanguageServices`.
Pyright provides some features that overlap with functionality provided by the standard VS Code Python extension: "hover", type completion, definitions, references, rename symbols, etc. You may see duplicate results if Pyright is installed alongside the Python extension. There is currently no way to disable this functionality in the Python extension. If you want to disable these features in Pyright, there is a setting to do so: `pyright.disableLanguageServices`.
## Documentation ## Documentation
* [Getting Started with Type Checking](/docs/getting-started.md)
* [Command-line Options](/docs/command-line.md)
* [Configuration](/docs/configuration.md)
* [Settings](/docs/settings.md)
* [Comments](/docs/comments.md)
* [Import Resolution](/docs/import-resolution.md)
* [Type Stubs](/docs/type-stubs.md)
* [Commands](/docs/commands.md)
* [Building & Debugging](/docs/build-debug.md)
* [Pyright Internals](/docs/internals.md)
- [Getting Started with Type Checking](/docs/getting-started.md)
- [Command-line Options](/docs/command-line.md)
- [Configuration](/docs/configuration.md)
- [Settings](/docs/settings.md)
- [Comments](/docs/comments.md)
- [Import Resolution](/docs/import-resolution.md)
- [Type Stubs](/docs/type-stubs.md)
- [Commands](/docs/commands.md)
- [Building & Debugging](/docs/build-debug.md)
- [Pyright Internals](/docs/internals.md)
## Limitations ## Limitations
Pyright currently provides support for Python 3.0 and newer. There is currently no plan to support older versions. Pyright currently provides support for Python 3.0 and newer. There is currently no plan to support older versions.
## Community ## Community
Do you have questions about Pyright or Python type annotations in general? Post your questions in this [gitter channel](https://gitter.im/microsoft-pyright/community). Do you have questions about Pyright or Python type annotations in general? Post your questions in this [gitter channel](https://gitter.im/microsoft-pyright/community).
## FAQ ## FAQ
**Q:** What is the difference between pyright and the [Microsoft Python Visual Studio Code plugin](https://github.com/Microsoft/vscode-python)? **Q:** What is the difference between pyright and the [Microsoft Python Visual Studio Code plugin](https://github.com/Microsoft/vscode-python)?
**A:** Pyright is focused on type checking. The Python VS Code plugin is Microsoft’s officially-supported extension for VS Code and provides a diverse array of features including debugging, test case management, linter plugins, and more. Pyright can be used alongside the Microsoft Python extension. **A:** Pyright is focused on type checking. The Python VS Code plugin is Microsoft's officially-supported extension for VS Code and provides a diverse array of features including debugging, test case management, linter plugins, and more. Pyright can be used alongside the Microsoft Python extension.
**Q:** What is the long-term plan for Pyright? **Q:** What is the long-term plan for Pyright?
**A:** Pyright is a side project with no dedicated team. There is no guarantee of continued development on the project. If you find it useful, feel free to use it and contribute to the code base. **A:** Pyright is a side project with no dedicated team. There is no guarantee of continued development on the project. If you find it useful, feel free to use it and contribute to the code base.
## Contributing ## Contributing
This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.microsoft.com. This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.microsoft.com.

42
client/README.md
View File

@ -1,35 +1,39 @@
# Static type checker for Python # Static type checker for Python
### Speed ### Speed
Pyright is a fast type checker meant for large Python source bases. It can run in a “watch” mode and performs fast incremental updates when files are modified.
Pyright is a fast type checker meant for large Python source bases. It can run in a "watch" mode and performs fast incremental updates when files are modified.
### Configurability ### Configurability
Pyright supports [configuration files](/docs/configuration.md) that provide granular control over settings. Different “execution environments” can be associated with subdirectories within a source base. Each environment can specify different module search paths, python language versions, and platform targets.
Pyright supports [configuration files](/docs/configuration.md) that provide granular control over settings. Different "execution environments" can be associated with subdirectories within a source base. Each environment can specify different module search paths, python language versions, and platform targets.
### Type Checking Features ### Type Checking Features
* [PEP 484](https://www.python.org/dev/peps/pep-0484/) type hints including generics
* [PEP 526](https://www.python.org/dev/peps/pep-0526/) syntax for variable annotations - [PEP 484](https://www.python.org/dev/peps/pep-0484/) type hints including generics
* [PEP 544](https://www.python.org/dev/peps/pep-0544/) structural subtyping - [PEP 526](https://www.python.org/dev/peps/pep-0526/) syntax for variable annotations
* [PEP 589](https://www.python.org/dev/peps/pep-0589/) typed dictionaries - [PEP 544](https://www.python.org/dev/peps/pep-0544/) structural subtyping
* Type inference for function return values, instance variables, class variables, and globals - [PEP 589](https://www.python.org/dev/peps/pep-0589/) typed dictionaries
* Smart type constraints that understand conditional code flow constructs like if/else statements - Type inference for function return values, instance variables, class variables, and globals
- Smart type constraints that understand conditional code flow constructs like if/else statements
### VS Code Language Features ### VS Code Language Features
The VS Code extension supports many time-saving language features including: The VS Code extension supports many time-saving language features including:
* Intelligent type completion of keywords, symbols, and import names appears when editing - Intelligent type completion of keywords, symbols, and import names appears when editing
* Import statements are automatically inserted when necessary for type completions - Import statements are automatically inserted when necessary for type completions
* Signature completion tips help when filling in arguments for a call - Signature completion tips help when filling in arguments for a call
* Hover over symbols to provide type information and doc strings - Hover over symbols to provide type information and doc strings
* Find Definitions to quickly go to the location of a symbol’s definition - Find Definitions to quickly go to the location of a symbol's definition
* Find References to find all references to a symbol within a code base - Find References to find all references to a symbol within a code base
* Rename Symbol to rename all references to a symbol within a code base - Rename Symbol to rename all references to a symbol within a code base
* Find Symbols within the current document or within the entire workspace - Find Symbols within the current document or within the entire workspace
* Organize Imports command for automatically ordering imports according to PEP8 rules - Organize Imports command for automatically ordering imports according to PEP8 rules
* Type stub generation for third-party libraries - Type stub generation for third-party libraries
### Built-in Type Stubs ### Built-in Type Stubs
Pyright includes a recent copy of the stdlib type stubs from [Typeshed](https://github.com/python/typeshed). It can be configured to use another (perhaps more recent or modified) copy of the Typeshed type stubs. Of course, it also works with custom type stub files that are part of your project. Pyright includes a recent copy of the stdlib type stubs from [Typeshed](https://github.com/python/typeshed). It can be configured to use another (perhaps more recent or modified) copy of the Typeshed type stubs. Of course, it also works with custom type stub files that are part of your project.
For more details, refer to the [README](https://github.com/Microsoft/pyright/blob/master/README.md) on the Pyright GitHub site. For more details, refer to the [README](https://github.com/Microsoft/pyright/blob/master/README.md) on the Pyright GitHub site.

46
client/package-lock.json generated
View File

@ -5,9 +5,9 @@
"requires": true, "requires": true,
"dependencies": { "dependencies": {
"@types/node": { "@types/node": {
"version": "12.12.14", "version": "13.7.4",
"resolved": "https://registry.npmjs.org/@types/node/-/node-12.12.14.tgz", "resolved": "https://registry.npmjs.org/@types/node/-/node-13.7.4.tgz",
"integrity": "sha512-u/SJDyXwuihpwjXy7hOOghagLEV1KdAST6syfnOk6QZAMzZuWZqXy5aYYZbh8Jdpd4escVFP0MvftHNDb9pruA==", "integrity": "sha512-oVeL12C6gQS/GAExndigSaLxTrKpQPxewx9bOcwfvJiJge4rr7wNaph4J+ns5hrmIV2as5qxqN8YKthn9qh0jw==",
"dev": true "dev": true
}, },
"agent-base": { "agent-base": {
@ -838,9 +838,9 @@
} }
}, },
"readable-stream": { "readable-stream": {
"version": "3.4.0", "version": "3.6.0",
"resolved": "https://registry.npmjs.org/readable-stream/-/readable-stream-3.4.0.tgz", "resolved": "https://registry.npmjs.org/readable-stream/-/readable-stream-3.6.0.tgz",
"integrity": "sha512-jItXPLmrSR8jmTRmRWJXCnGJsfy85mB3Wd/uINMXA65yrnFo0cPClFIUWzo2najVNSl+mx7/4W8ttlLWJe99pQ==", "integrity": "sha512-BViHy7LKeTz4oNnkcLJ+lVSL6vpiFeX6/d3oSH8zCW7UxP2onchk+vTGB143xuFjHS3deTgkKoXXymXqymiIdA==",
"dev": true, "dev": true,
"requires": { "requires": {
"inherits": "^2.0.3", "inherits": "^2.0.3",
@ -1016,9 +1016,9 @@
} }
}, },
"typescript": { "typescript": {
"version": "3.7.3", "version": "3.8.2",
"resolved": "https://registry.npmjs.org/typescript/-/typescript-3.7.3.tgz", "resolved": "https://registry.npmjs.org/typescript/-/typescript-3.8.2.tgz",
"integrity": "sha512-Mcr/Qk7hXqFBXMN7p7Lusj1ktCBydylfQM/FZCk5glCNQJrCUKPkMHdo9R0MTFWsC/4kPFvDS0fDPvukfCkFsw==", "integrity": "sha512-EgOVgL/4xfVrCMbhYKUQTdF37SQn4Iw73H5BgCrF1Abdun7Kwy/QZsE/ssAy0y4LxBbvua3PIbFsbRczWWnDdQ==",
"dev": true "dev": true
}, },
"uc.micro": { "uc.micro": {
@ -1082,9 +1082,9 @@
} }
}, },
"vsce": { "vsce": {
"version": "1.70.0", "version": "1.73.0",
"resolved": "https://registry.npmjs.org/vsce/-/vsce-1.70.0.tgz", "resolved": "https://registry.npmjs.org/vsce/-/vsce-1.73.0.tgz",
"integrity": "sha512-mBTbVrWL348jODwfmaR+yXrlzb8EABGCT067C4shKOXriWiuMQi4/uCbFm6TUBcfnzTYLJv+DKa0VnKU8yEAjA==", "integrity": "sha512-6W37Ebbkj3uF3WhT+SCfRtsneRQEFcGvf/XYz+b6OAgDCj4gPurWyDVrqw/HLsbP1WflGIyUfVZ8t5M7kQp6Uw==",
"dev": true, "dev": true,
"requires": { "requires": {
"azure-devops-node-api": "^7.2.0", "azure-devops-node-api": "^7.2.0",
@ -1125,9 +1125,9 @@
} }
}, },
"vscode-jsonrpc": { "vscode-jsonrpc": {
"version": "5.0.0-next.2", "version": "5.0.1",
"resolved": "https://registry.npmjs.org/vscode-jsonrpc/-/vscode-jsonrpc-5.0.0-next.2.tgz", "resolved": "https://registry.npmjs.org/vscode-jsonrpc/-/vscode-jsonrpc-5.0.1.tgz",
"integrity": "sha512-Q3/jabZUNviCG9hhF6hHWjhrABevPF9mv0aiE2j8BYCAP2k+aHTpjMyk+04MzaAqWYwXdQuZkLSbcYCCqbzJLg==" "integrity": "sha512-JvONPptw3GAQGXlVV2utDcHx0BiY34FupW/kI6mZ5x06ER5DdPG/tXWMVHjTNULF5uKPOUUD0SaXg5QaubJL0A=="
}, },
"vscode-languageclient": { "vscode-languageclient": {
"version": "5.3.0-next.9", "version": "5.3.0-next.9",
@ -1155,18 +1155,18 @@
} }
}, },
"vscode-languageserver-protocol": { "vscode-languageserver-protocol": {
"version": "3.15.0-next.9", "version": "3.15.3",
"resolved": "https://registry.npmjs.org/vscode-languageserver-protocol/-/vscode-languageserver-protocol-3.15.0-next.9.tgz", "resolved": "https://registry.npmjs.org/vscode-languageserver-protocol/-/vscode-languageserver-protocol-3.15.3.tgz",
"integrity": "sha512-b9PAxouMmtsLEe8ZjbIMPb7wRWPhckGfgjwZLmp/dWnaAuRPYtY3lGO0/rNbLc3jKIqCVlnEyYVFKalzDAzj0g==", "integrity": "sha512-zrMuwHOAQRhjDSnflWdJG+O2ztMWss8GqUUB8dXLR/FPenwkiBNkMIJJYfSN6sgskvsF0rHAoBowNQfbyZnnvw==",
"requires": { "requires": {
"vscode-jsonrpc": "^5.0.0-next.2", "vscode-jsonrpc": "^5.0.1",
"vscode-languageserver-types": "^3.15.0-next.5" "vscode-languageserver-types": "3.15.1"
} }
}, },
"vscode-languageserver-types": { "vscode-languageserver-types": {
"version": "3.15.0-next.5", "version": "3.15.1",
"resolved": "https://registry.npmjs.org/vscode-languageserver-types/-/vscode-languageserver-types-3.15.0-next.5.tgz", "resolved": "https://registry.npmjs.org/vscode-languageserver-types/-/vscode-languageserver-types-3.15.1.tgz",
"integrity": "sha512-7hrELhTeWieUgex3+6692KjCkcmO/+V/bFItM5MHGcBotzwmjEuXjapLLYTYhIspuJ1ibRSik5MhX5YwLpsPiw==" "integrity": "sha512-+a9MPUQrNGRrGU630OGbYVQ+11iOIovjCkqxajPa9w57Sd5ruK8WQNsslzpa0x/QJqC8kRc2DUxWjIFwoNm4ZQ=="
}, },
"vscode-test": { "vscode-test": {
"version": "0.4.3", "version": "0.4.3",

4
client/package.json
View File

@ -116,8 +116,8 @@
"postinstall": "node ./node_modules/vscode/bin/install" "postinstall": "node ./node_modules/vscode/bin/install"
}, },
"devDependencies": { "devDependencies": {
"typescript": "^3.7.3", "typescript": "^3.8.2",
"vsce": "^1.70.0", "vsce": "^1.73.0",
"vscode": "^1.1.36" "vscode": "^1.1.36"
}, },
"dependencies": { "dependencies": {

15
client/schemas/pyrightconfig.schema.json
View File

@ -5,8 +5,17 @@
"definitions": { "definitions": {
"diagnostic": { "diagnostic": {
"anyOf": [ "anyOf": [
{ "type": "boolean" }, {
{ "type": "string", "enum": [ "none", "warning", "error" ] } "type": "boolean"
},
{
"type": "string",
"enum": [
"none",
"warning",
"error"
]
}
] ]
} }
}, },
@ -93,7 +102,7 @@
"enableTypeIgnoreComments": { "enableTypeIgnoreComments": {
"$id": "#/properties/enableTypeIgnoreComments", "$id": "#/properties/enableTypeIgnoreComments",
"type": "boolean", "type": "boolean",
"title": "Allow “# type: ignore” comments", "title": "Allow \"# type: ignore\" comments",
"default": "true" "default": "true"
}, },
"reportTypeshedErrors": { "reportTypeshedErrors": {

185
client/src/extension.ts
View File

@ -1,102 +1,117 @@
/* /*
* extension.ts * extension.ts
* *
* Copyright (c) Microsoft Corporation. * Copyright (c) Microsoft Corporation.
* Licensed under the MIT license. * Licensed under the MIT license.
* *
* Provides client for Pyright Python language server. This portion runs * Provides client for Pyright Python language server. This portion runs
* in the context of the VS Code process and talks to the server, which * in the context of the VS Code process and talks to the server, which
* runs in another process. * runs in another process.
*/ */
import * as fs from 'fs'; import * as fs from 'fs';
import * as path from 'path'; import * as path from 'path';
import { commands, ExtensionContext, Position, Range, TextEditor, TextEditorEdit } from 'vscode'; import { commands, ExtensionContext, Position, Range, TextEditor, TextEditorEdit } from 'vscode';
import { LanguageClient, LanguageClientOptions, ServerOptions, TextEdit,TransportKind } from 'vscode-languageclient'; import { LanguageClient, LanguageClientOptions, ServerOptions, TextEdit, TransportKind } from 'vscode-languageclient';
import { Commands } from '../../server/src/commands/commands'; import { Commands } from '../../server/src/commands/commands';
import { ProgressReporting } from './progress'; import { ProgressReporting } from './progress';
export function activate(context: ExtensionContext) { export function activate(context: ExtensionContext) {
const bundlePath = context.asAbsolutePath(path.join('server', 'server.bundle.js')); const bundlePath = context.asAbsolutePath(path.join('server', 'server.bundle.js'));
const nonBundlePath = context.asAbsolutePath(path.join('server', 'src', 'server.js')); const nonBundlePath = context.asAbsolutePath(path.join('server', 'src', 'server.js'));
const debugOptions = { execArgv: ["--nolazy", "--inspect=6600"] }; const debugOptions = { execArgv: ['--nolazy', '--inspect=6600'] };
// If the extension is launched in debug mode, then the debug server options are used.
const serverOptions: ServerOptions = {
run : { module: bundlePath, transport: TransportKind.ipc },
// In debug mode, use the non-bundled code if it's present. The production
// build includes only the bundled package, so we don't want to crash if
// someone starts the production extension in debug mode.
debug: { module: fs.existsSync(nonBundlePath) ? nonBundlePath : bundlePath,
transport: TransportKind.ipc, options: debugOptions }
}
// Options to control the language client
const clientOptions: LanguageClientOptions = {
// Register the server for python source files.
documentSelector: [{
scheme: 'file',
language: 'python'
}],
synchronize: {
// Synchronize the setting section to the server.
configurationSection: ['python', 'pyright']
}
}
// Create the language client and start the client.
const languageClient = new LanguageClient('python', 'Pyright', serverOptions, clientOptions);
const disposable = languageClient.start();
// Push the disposable to the context's subscriptions so that the
// client can be deactivated on extension deactivation.
context.subscriptions.push(disposable);
// Allocate a progress reporting object. // If the extension is launched in debug mode, then the debug server options are used.
const progressReporting = new ProgressReporting(languageClient); const serverOptions: ServerOptions = {
context.subscriptions.push(progressReporting); run: { module: bundlePath, transport: TransportKind.ipc },
// In debug mode, use the non-bundled code if it's present. The production
// build includes only the bundled package, so we don't want to crash if
// someone starts the production extension in debug mode.
debug: {
module: fs.existsSync(nonBundlePath) ? nonBundlePath : bundlePath,
transport: TransportKind.ipc,
options: debugOptions
}
};
// Register our custom commands. // Options to control the language client
const textEditorCommands = [Commands.orderImports, Commands.addMissingOptionalToParam]; const clientOptions: LanguageClientOptions = {
textEditorCommands.forEach(commandName => { // Register the server for python source files.
context.subscriptions.push(commands.registerTextEditorCommand(commandName, documentSelector: [
(editor: TextEditor, edit: TextEditorEdit, ...args: any[]) => { {
const cmd = { scheme: 'file',
command: commandName, language: 'python'
arguments: [editor.document.uri.toString(), ...args] }
}; ],
synchronize: {
languageClient.sendRequest('workspace/executeCommand', cmd).then((edits: TextEdit[] | undefined) => { // Synchronize the setting section to the server.
if (edits && edits.length > 0) { configurationSection: ['python', 'pyright']
editor.edit(editBuilder => { }
edits.forEach(edit => { };
const startPos = new Position(edit.range.start.line, edit.range.start.character);
const endPos = new Position(edit.range.end.line, edit.range.end.character);
const range = new Range(startPos, endPos);
editBuilder.replace(range, edit.newText);
});
});
}
});
},
() => {
// Error received. For now, do nothing.
}));
});
const genericCommands = [ Commands.createTypeStub, ]; // Create the language client and start the client.
genericCommands.forEach(command => { const languageClient = new LanguageClient('python', 'Pyright', serverOptions, clientOptions);
context.subscriptions.push(commands.registerCommand(command, (...args: any[]) => { const disposable = languageClient.start();
languageClient.sendRequest('workspace/executeCommand', { command, arguments: args });
})); // Push the disposable to the context's subscriptions so that the
}); // client can be deactivated on extension deactivation.
context.subscriptions.push(disposable);
// Allocate a progress reporting object.
const progressReporting = new ProgressReporting(languageClient);
context.subscriptions.push(progressReporting);
// Register our custom commands.
const textEditorCommands = [Commands.orderImports, Commands.addMissingOptionalToParam];
textEditorCommands.forEach(commandName => {
context.subscriptions.push(
commands.registerTextEditorCommand(
commandName,
(editor: TextEditor, edit: TextEditorEdit, ...args: any[]) => {
const cmd = {
command: commandName,
arguments: [editor.document.uri.toString(), ...args]
};
languageClient
.sendRequest('workspace/executeCommand', cmd)
.then((edits: TextEdit[] | undefined) => {
if (edits && edits.length > 0) {
editor.edit(editBuilder => {
edits.forEach(edit => {
const startPos = new Position(
edit.range.start.line,
edit.range.start.character
);
const endPos = new Position(edit.range.end.line, edit.range.end.character);
const range = new Range(startPos, endPos);
editBuilder.replace(range, edit.newText);
});
});
}
});
},
() => {
// Error received. For now, do nothing.
}
)
);
});
const genericCommands = [Commands.createTypeStub];
genericCommands.forEach(command => {
context.subscriptions.push(
commands.registerCommand(command, (...args: any[]) => {
languageClient.sendRequest('workspace/executeCommand', { command, arguments: args });
})
);
});
} }
export function deactivate() { export function deactivate() {
// Return undefined rather than a promise to indicate // Return undefined rather than a promise to indicate
// that deactivation is done synchronously. We don't have // that deactivation is done synchronously. We don't have
// anything to do here. // anything to do here.
return undefined; return undefined;
} }

33
client/src/progress.ts
View File

@ -1,12 +1,12 @@
/* /*
* progress.ts * progress.ts
* *
* Copyright (c) Microsoft Corporation. * Copyright (c) Microsoft Corporation.
* Licensed under the MIT license. * Licensed under the MIT license.
* *
* Provides a way for the pyright language server to report progress * Provides a way for the pyright language server to report progress
* back to the client and display it in the editor. * back to the client and display it in the editor.
*/ */
import { Progress, ProgressLocation, window } from 'vscode'; import { Progress, ProgressLocation, window } from 'vscode';
import { Disposable, LanguageClient } from 'vscode-languageclient'; import { Disposable, LanguageClient } from 'vscode-languageclient';
@ -25,13 +25,16 @@ export class ProgressReporting implements Disposable {
this._resolveProgress = resolve; this._resolveProgress = resolve;
}); });
window.withProgress({ window.withProgress(
location: ProgressLocation.Window, {
title: '' location: ProgressLocation.Window,
}, progress => { title: ''
this._progress = progress; },
return progressPromise; progress => {
}); this._progress = progress;
return progressPromise;
}
);
this._primeTimeoutTimer(); this._primeTimeoutTimer();
}); });

11
docs/build-debug.md
View File

@ -1,6 +1,7 @@
## Building Pyright ## Building Pyright
To build the project: To build the project:
1. Install [nodejs](https://nodejs.org/en/) 1. Install [nodejs](https://nodejs.org/en/)
2. Open terminal window in main directory of cloned source 2. Open terminal window in main directory of cloned source
3. Execute `npm run install:all` to install dependencies 3. Execute `npm run install:all` to install dependencies
@ -8,11 +9,11 @@ To build the project:
To build the VS Code extension package: To build the VS Code extension package:
Same as above, plus Same as above, plus
1. Execute `npm run package` 1. Execute `npm run package`
The resulting package (pyright-X.Y.Z.vsix) can be found in the client directory. The resulting package (pyright-X.Y.Z.vsix) can be found in the client directory.
To install in VS Code, go to the extensions panel and choose “Install from VSIX...” from the menu, then select the package. To install in VS Code, go to the extensions panel and choose "Install from VSIX..." from the menu, then select the package.
## Running Pyright Locally ## Running Pyright Locally
@ -20,10 +21,8 @@ Once built, you can run the command-line tool directly from the built sources by
`node ./index.js` `node ./index.js`
## Debugging Pyright ## Debugging Pyright
To debug pyright, open the root source directory within VS Code. Open the debug sub-panel and choose “Pyright CLI” from the debug target menu. Click on the green “run” icon or press F5 to build and launch the command-line version in the VS Code debugger. To debug pyright, open the root source directory within VS Code. Open the debug sub-panel and choose "Pyright CLI" from the debug target menu. Click on the green "run" icon or press F5 to build and launch the command-line version in the VS Code debugger.
To debug the VS Code extension, select “Pyright Language Client” from the debug target menu. Click on the green “run” icon or press F5 to build and launch a second copy of VS Code with the extension. Within the second VS Code instance, open a python source file so the pyright extension is loaded. Return to the first instance of VS Code and select “Pyright Language Server” from the debug target menu and click the green “run” icon. This will attach the debugger to the process that hosts the type checker. You can now set breakpoints, etc.
To debug the VS Code extension, select "Pyright Language Client" from the debug target menu. Click on the green "run" icon or press F5 to build and launch a second copy of VS Code with the extension. Within the second VS Code instance, open a python source file so the pyright extension is loaded. Return to the first instance of VS Code and select "Pyright Language Server" from the debug target menu and click the green "run" icon. This will attach the debugger to the process that hosts the type checker. You can now set breakpoints, etc.

47
docs/command-line.md
View File

@ -2,35 +2,32 @@
Pyright can be run as either a VS Code extension or as a node-based command-line tool. The command-line version allows for the following options: Pyright can be run as either a VS Code extension or as a node-based command-line tool. The command-line version allows for the following options:
| Flag | Description | | Flag | Description |
| :--------------------------------- | :--------------------------------------------------- | | :------------------------------ | :------------------------------------------------ |
| --createstub IMPORT | Create type stub file(s) for import | | --createstub IMPORT | Create type stub file(s) for import |
| --dependencies | Emit import dependency information | | --dependencies | Emit import dependency information |
| -h, --help | Show help message | | -h, --help | Show help message |
| --lib | Use library code for types when stubs are missing | | --lib | Use library code for types when stubs are missing |
| --outputjson | Output results in JSON format | | --outputjson | Output results in JSON format |
| -p, --project FILE OR DIRECTORY | Use the configuration file at this location | | -p, --project FILE OR DIRECTORY | Use the configuration file at this location |
| --stats | Print detailed performance stats | | --stats | Print detailed performance stats |
| -t, --typeshed-path DIRECTORY | Use typeshed type stubs at this location (1) | | -t, --typeshed-path DIRECTORY | Use typeshed type stubs at this location (1) |
| -v, --venv-path DIRECTORY | Directory that contains virtual environments (2) | | -v, --venv-path DIRECTORY | Directory that contains virtual environments (2) |
| --verbose | Emit verbose diagnostics | | --verbose | Emit verbose diagnostics |
| --version | Print pyright version | | --version | Print pyright version |
| -w, --watch | Continue to run and watch for changes (3) | | -w, --watch | Continue to run and watch for changes (3) |
(1) Pyright has built-in typeshed type stubs for Python stdlib functionality. To use a different version of typeshed type stubs, specify the directory with this option. (1) Pyright has built-in typeshed type stubs for Python stdlib functionality. To use a different version of typeshed type stubs, specify the directory with this option.
(2) This option is used in conjunction with configuration file, which can refer to different virtual environments by name. For more details, refer to the [configuration](/docs/configuration.md) documentation. This allows a common config file to be checked in to the project and shared by everyone on the development team without making assumptions about the local paths to the venv directory on each developer’s computer. (2) This option is used in conjunction with configuration file, which can refer to different virtual environments by name. For more details, refer to the [configuration](/docs/configuration.md) documentation. This allows a common config file to be checked in to the project and shared by everyone on the development team without making assumptions about the local paths to the venv directory on each developer's computer.
(3) When running in watch mode, pyright will reanalyze only those files that have been modified. These “deltas” are typically much faster than the initial analysis, which needs to analyze all files in the source tree.
(3) When running in watch mode, pyright will reanalyze only those files that have been modified. These "deltas" are typically much faster than the initial analysis, which needs to analyze all files in the source tree.
# Pyright Exit Codes # Pyright Exit Codes
| Exit Code | Meaning | | Exit Code | Meaning |
| :---------- | :--------------------------------------------------------------- | | :-------- | :------------------------------------------------------- |
| 0 | No errors reported | | 0 | No errors reported |
| 1 | One or more errors reported | | 1 | One or more errors reported |
| 2 | Fatal error occurred with no errors or warnings reported | | 2 | Fatal error occurred with no errors or warnings reported |
| 3 | Config file could not be read or parsed | | 3 | Config file could not be read or parsed |

5
docs/commands.md
View File

@ -1,8 +1,9 @@
# VS Code Commands # VS Code Commands
Pyright offers the following commands, which can be invoked from VS Code’s “Command Palette”, which can be accessed from the View menu or by pressing Cmd-Shift-P. Pyright offers the following commands, which can be invoked from VS Code's "Command Palette", which can be accessed from the View menu or by pressing Cmd-Shift-P.
## Organize Imports ## Organize Imports
This command reorders all imports found in the global (module-level) scope of the source file. As recommended in PEP8, imports are grouped into three groups, each separated by an empty line. The first group includes all built-in modules, the second group includes all third-party modules, and the third group includes all local modules. This command reorders all imports found in the global (module-level) scope of the source file. As recommended in PEP8, imports are grouped into three groups, each separated by an empty line. The first group includes all built-in modules, the second group includes all third-party modules, and the third group includes all local modules.
Within each group, imports are sorted alphabetically. And within each “from X import Y” statement, the imported symbols are sorted alphabetically. Pyright also rewraps any imports that don't fit within a single line, switching to multi-line formatting. Within each group, imports are sorted alphabetically. And within each "from X import Y" statement, the imported symbols are sorted alphabetically. Pyright also rewraps any imports that don't fit within a single line, switching to multi-line formatting.

5
docs/comments.md
View File

@ -3,6 +3,7 @@
Some behaviors of pyright can be controlled through the use of comments within the source file. Some behaviors of pyright can be controlled through the use of comments within the source file.
## Type Annotations ## Type Annotations
Versions of Python prior to 3.6 did not support type annotations for variables. Pyright honors type annotations found within a comment at the end of the same line where a variable is assigned. Versions of Python prior to 3.6 did not support type annotations for variables. Pyright honors type annotations found within a comment at the end of the same line where a variable is assigned.
``` ```
@ -12,13 +13,14 @@ self._target = 3 # type: Union[int, str]
``` ```
## File-level Type Controls ## File-level Type Controls
Strict typing controls (where all supported type-checking switches generate errors) can be enabled for a file through the use of a special comment. Typically this comment is placed at or near the top of a code file on its own line. Strict typing controls (where all supported type-checking switches generate errors) can be enabled for a file through the use of a special comment. Typically this comment is placed at or near the top of a code file on its own line.
``` ```
# pyright: strict # pyright: strict
``` ```
Individual configuration settings can also be overridden on a per-file basis and combined with “strict” typing. For example, if you want to enable all type checks except for “reportPrivateUsage”, you could add the following comment: Individual configuration settings can also be overridden on a per-file basis and combined with "strict" typing. For example, if you want to enable all type checks except for "reportPrivateUsage", you could add the following comment:
``` ```
# pyright: strict, reportPrivateUsage=false # pyright: strict, reportPrivateUsage=false
@ -29,4 +31,3 @@ Diagnostic levels are also supported.
``` ```
# pyright: reportPrivateUsage=warning, reportOptionalCall=error # pyright: reportPrivateUsage=warning, reportOptionalCall=error
``` ```

118
docs/configuration.md
View File

@ -1,24 +1,24 @@
# Pyright Configuration # Pyright Configuration
Pyright offers flexible configuration options specified in a JSON-formatted text configuration. By default, the file is called “pyrightconfig.json” and is located within the root directory of your project. Multi-root workspaces (“Add Folder to Workspace…”) are supported, and each workspace root can have its own “pyrightconfig.json” file. Pyright offers flexible configuration options specified in a JSON-formatted text configuration. By default, the file is called "pyrightconfig.json" and is located within the root directory of your project. Multi-root workspaces ("Add Folder to Workspace...") are supported, and each workspace root can have its own "pyrightconfig.json" file.
Relative paths specified within the config file are relative to the config file’s location. Paths with shell variables (including `~`) are not supported. Relative paths specified within the config file are relative to the config file's location. Paths with shell variables (including `~`) are not supported.
## Master Pyright Config Options ## Master Pyright Config Options
**include** [array of paths, optional]: Paths of directories or files that should be included. If no paths are specified, pyright defaults to the directory that contains the config file. Paths may contain wildcard characters ** (a directory or multiple levels of directories), * (a sequence of zero or more characters), or ? (a single character). If no include paths are specified, the root path for the workspace is assumed. **include** [array of paths, optional]: Paths of directories or files that should be included. If no paths are specified, pyright defaults to the directory that contains the config file. Paths may contain wildcard characters \*_ (a directory or multiple levels of directories), _ (a sequence of zero or more characters), or ? (a single character). If no include paths are specified, the root path for the workspace is assumed.
**exclude** [array of paths, optional]: Paths of directories or files that should not be included. These override the includes directories, allowing specific subdirectories to be ignored. Note that files in the exclude paths may still be included in the analysis if they are referenced (imported) by source files that are not excluded. Paths may contain wildcard characters ** (a directory or multiple levels of directories), * (a sequence of zero or more characters), or ? (a single character). If no exclude paths are specified, Pyright automatically excludes the following: `**/node_modules`, `**/__pycache__`, `.venv` and `.git`. **exclude** [array of paths, optional]: Paths of directories or files that should not be included. These override the includes directories, allowing specific subdirectories to be ignored. Note that files in the exclude paths may still be included in the analysis if they are referenced (imported) by source files that are not excluded. Paths may contain wildcard characters ** (a directory or multiple levels of directories), \* (a sequence of zero or more characters), or ? (a single character). If no exclude paths are specified, Pyright automatically excludes the following: `**/node_modules`,`\*\*/**pycache**`,`.venv`and`.git`.
**ignore** [array of paths, optional]: Paths of directories or files whose diagnostic output (errors and warnings) should be suppressed even if they are an included file or within the transitive closure of an included file. Paths may contain wildcard characters ** (a directory or multiple levels of directories), * (a sequence of zero or more characters), or ? (a single character). **ignore** [array of paths, optional]: Paths of directories or files whose diagnostic output (errors and warnings) should be suppressed even if they are an included file or within the transitive closure of an included file. Paths may contain wildcard characters `**` (a directory or multiple levels of directories), `*` (a sequence of zero or more characters), or `?` (a single character).
**strict** [array of paths, optional]: Paths of directories or files that should use “strict” analysis if they are included. This is the same as manually adding a “# pyright: strict” comment. In strict mode, all type-checking rules are enabled. Paths may contain wildcard characters ** (a directory or multiple levels of directories), * (a sequence of zero or more characters), or ? (a single character). **strict** [array of paths, optional]: Paths of directories or files that should use "strict" analysis if they are included. This is the same as manually adding a "# pyright: strict" comment. In strict mode, all type-checking rules are enabled. Paths may contain wildcard characters `**` (a directory or multiple levels of directories), `*` (a sequence of zero or more characters), or `?` (a single character).
**typeshedPath** [path, optional]: Path to a directory that contains typeshed type stub files. Pyright ships with an internal copy of some typeshed type stubs (those that cover the Python stdlib packages). If you want to use a full copy of the typeshed type stubs (including those for third-party packages), you can clone the [typeshed github repo](https://github.com/python/typeshed) to a local directory and reference the location with this path. **typeshedPath** [path, optional]: Path to a directory that contains typeshed type stub files. Pyright ships with an internal copy of some typeshed type stubs (those that cover the Python stdlib packages). If you want to use a full copy of the typeshed type stubs (including those for third-party packages), you can clone the [typeshed github repo](https://github.com/python/typeshed) to a local directory and reference the location with this path.
**typingsPath** [path, optional]: Path to a directory that contains custom type stubs. Each package's type stub file(s) are expected to be in its own subdirectory. The default value of this setting is "./typings". **typingsPath** [path, optional]: Path to a directory that contains custom type stubs. Each package's type stub file(s) are expected to be in its own subdirectory. The default value of this setting is "./typings".
**venvPath** [path, optional]: Path to a directory containing one or more subdirectories, each of which contains a virtual environment. Each execution environment (see below for details) can refer to a different virtual environment. When used in conjunction with a **venv** setting (see below), pyright will search for imports in the virtual environment’s site-packages directory rather than the paths specified in PYTHONPATH. **venvPath** [path, optional]: Path to a directory containing one or more subdirectories, each of which contains a virtual environment. Each execution environment (see below for details) can refer to a different virtual environment. When used in conjunction with a **venv** setting (see below), pyright will search for imports in the virtual environment's site-packages directory rather than the paths specified in PYTHONPATH.
**venv** [string, optional]: Used in conjunction with the venvPath, specifies the virtual environment to use. Individual execution environments may override this setting. **venv** [string, optional]: Used in conjunction with the venvPath, specifies the virtual environment to use. Individual execution environments may override this setting.
@ -28,19 +28,19 @@ Relative paths specified within the config file are relative to the config file
**executionEnvironments** [array of objects, optional]: Specifies a list of execution environments (see below). Execution environments are searched from start to finish by comparing the path of a source file with the root path specified in the execution environment. **executionEnvironments** [array of objects, optional]: Specifies a list of execution environments (see below). Execution environments are searched from start to finish by comparing the path of a source file with the root path specified in the execution environment.
## Type Check Diagnostics Settings ## Type Check Diagnostics Settings
The following settings control pyright’s diagnostic output (warnings or errors). Unless otherwise specified, each diagnostic setting can specify a boolean value (`false` indicating that no error is generated and `true` indicating that an error is generated). Alternatively, a string value of `"none"`, `"warning"`, or `"error"` can be used to specify the diagnostic level.
The following settings control pyright's diagnostic output (warnings or errors). Unless otherwise specified, each diagnostic setting can specify a boolean value (`false` indicating that no error is generated and `true` indicating that an error is generated). Alternatively, a string value of `"none"`, `"warning"`, or `"error"` can be used to specify the diagnostic level.
**strictListInference** [boolean]: When inferring the type of a list, use strict type assumptions. For example, the expression `[1, 'a', 3.4]` could be inferred to be of type `List[Any]` or `List[Union[int, str, float]]`. If this setting is true, it will use the latter (stricter) type. The default value for this setting is 'false'. **strictListInference** [boolean]: When inferring the type of a list, use strict type assumptions. For example, the expression `[1, 'a', 3.4]` could be inferred to be of type `List[Any]` or `List[Union[int, str, float]]`. If this setting is true, it will use the latter (stricter) type. The default value for this setting is 'false'.
**strictDictionaryInference** [boolean]: When inferring the type of a dictionary’s keys and values, use strict type assumptions. For example, the expression `{'a': 1, 'b': 'a'}` could be inferred to be of type `Dict[str, Any]` or `Dict[str, Union[int, str]]`. If this setting is true, it will use the latter (stricter) type. The default value for this setting is 'false'. **strictDictionaryInference** [boolean]: When inferring the type of a dictionary's keys and values, use strict type assumptions. For example, the expression `{'a': 1, 'b': 'a'}` could be inferred to be of type `Dict[str, Any]` or `Dict[str, Union[int, str]]`. If this setting is true, it will use the latter (stricter) type. The default value for this setting is 'false'.
**strictParameterNoneValue** [boolean]: PEP 484 indicates that when a function parameter is assigned a default value of None, its type should implicitly be Optional even if the explicit type is not. When enabled, this rule requires that parameter type annotations use Optional explicitly in this case. The default value for this setting is 'false'. **strictParameterNoneValue** [boolean]: PEP 484 indicates that when a function parameter is assigned a default value of None, its type should implicitly be Optional even if the explicit type is not. When enabled, this rule requires that parameter type annotations use Optional explicitly in this case. The default value for this setting is 'false'.
**enableTypeIgnoreComments** [boolean]: PEP 484 defines support for "# type: ignore" comments. This switch enables or disables support for these comments. The default value for this setting is 'true'. **enableTypeIgnoreComments** [boolean]: PEP 484 defines support for "# type: ignore" comments. This switch enables or disables support for these comments. The default value for this setting is 'true'.
**reportTypeshedErrors** [boolean or string, optional]: Generate or suppress diagnostics for typeshed type stub files. In general, these type stub files should be “clean” and generate no errors. The default value for this setting is 'none'. **reportTypeshedErrors** [boolean or string, optional]: Generate or suppress diagnostics for typeshed type stub files. In general, these type stub files should be "clean" and generate no errors. The default value for this setting is 'none'.
**reportMissingImports** [boolean or string, optional]: Generate or suppress diagnostics for imports that have no corresponding imported python file or type stub file. The default value for this setting is 'none', although pyright can do a much better job of static type checking if type stub files are provided for all imports. **reportMissingImports** [boolean or string, optional]: Generate or suppress diagnostics for imports that have no corresponding imported python file or type stub file. The default value for this setting is 'none', although pyright can do a much better job of static type checking if type stub files are provided for all imports.
@ -76,9 +76,9 @@ The following settings control pyright’s diagnostic output (warnings or errors
**reportUntypedBaseClass** [boolean or string, optional]: Generate or suppress diagnostics for base classes whose type cannot be determined statically. These obscure the class type, defeating many type analysis features. The default value for this setting is 'none'. **reportUntypedBaseClass** [boolean or string, optional]: Generate or suppress diagnostics for base classes whose type cannot be determined statically. These obscure the class type, defeating many type analysis features. The default value for this setting is 'none'.
**reportUntypedNamedTuple** [boolean or string, optional]: Generate or suppress diagnostics when “namedtuple” is used rather than “NamedTuple”. The former contains no type information, whereas the latter does. The default value for this setting is 'none'. **reportUntypedNamedTuple** [boolean or string, optional]: Generate or suppress diagnostics when "namedtuple" is used rather than "NamedTuple". The former contains no type information, whereas the latter does. The default value for this setting is 'none'.
**reportPrivateUsage** [boolean or string, optional]: Generate or suppress diagnostics for incorrect usage of private or protected variables or functions. Protected class members begin with a single underscore (“_”) and can be accessed only by subclasses. Private class members begin with a double underscore but do not end in a double underscore and can be accessed only within the declaring class. Variables and functions declared outside of a class are considered private if their names start with either a single or double underscore, and they cannot be accessed outside of the declaring module. The default value for this setting is 'none'. **reportPrivateUsage** [boolean or string, optional]: Generate or suppress diagnostics for incorrect usage of private or protected variables or functions. Protected class members begin with a single underscore ("\_") and can be accessed only by subclasses. Private class members begin with a double underscore but do not end in a double underscore and can be accessed only within the declaring class. Variables and functions declared outside of a class are considered private if their names start with either a single or double underscore, and they cannot be accessed outside of the declaring module. The default value for this setting is 'none'.
**reportConstantRedefinition** [boolean or string, optional]: Generate or suppress diagnostics for attempts to redefine variables whose names are all-caps with underscores and numerals. The default value for this setting is 'none'. **reportConstantRedefinition** [boolean or string, optional]: Generate or suppress diagnostics for attempts to redefine variables whose names are all-caps with underscores and numerals. The default value for this setting is 'none'.
@ -104,13 +104,13 @@ The following settings control pyright’s diagnostic output (warnings or errors
**reportAssertAlwaysTrue** [boolean or string, optional]: Generate or suppress diagnostics for 'assert' statement that will provably always assert. This can be indicative of a programming error. The default value for this setting is 'warning'. **reportAssertAlwaysTrue** [boolean or string, optional]: Generate or suppress diagnostics for 'assert' statement that will provably always assert. This can be indicative of a programming error. The default value for this setting is 'warning'.
**reportSelfClsParameterName** [boolean or string, optional]: Generate or suppress diagnostics for a missing or misnamed “self” parameter in instance methods and “cls” parameter in class methods. Instance methods in metaclasses (classes that derive from “type”) are allowed to use “cls” for instance methods. The default value for this setting is 'warning'. **reportSelfClsParameterName** [boolean or string, optional]: Generate or suppress diagnostics for a missing or misnamed "self" parameter in instance methods and "cls" parameter in class methods. Instance methods in metaclasses (classes that derive from "type") are allowed to use "cls" for instance methods. The default value for this setting is 'warning'.
**reportImplicitStringConcatenation** [boolean or string, optional]: Generate or suppress diagnostics for two or more string literals that follow each other, indicating an implicit concatenation. This is considered a bad practice and often masks bugs such as missing commas. The default value for this setting is 'none'. **reportImplicitStringConcatenation** [boolean or string, optional]: Generate or suppress diagnostics for two or more string literals that follow each other, indicating an implicit concatenation. This is considered a bad practice and often masks bugs such as missing commas. The default value for this setting is 'none'.
## Execution Environment Options ## Execution Environment Options
Pyright allows multiple “execution environments” to be defined for different portions of your source tree. For example, a subtree may be designed to run with different import search paths or a different version of the python interpreter than the rest of the source base.
Pyright allows multiple "execution environments" to be defined for different portions of your source tree. For example, a subtree may be designed to run with different import search paths or a different version of the python interpreter than the rest of the source base.
The following settings can be specified for each execution environment. The following settings can be specified for each execution environment.
@ -124,72 +124,56 @@ The following settings can be specified for each execution environment.
**pythonPlatform** [string, optional]: Specifies the target platform that will be used for this execution environment. If not specified, the global `pythonPlatform` setting is used instead. **pythonPlatform** [string, optional]: Specifies the target platform that will be used for this execution environment. If not specified, the global `pythonPlatform` setting is used instead.
# VS Code Extension Settings # VS Code Extension Settings
Pyright will import the following settings set through VS Code. These override the values provided in the configuration file. Pyright will import the following settings set through VS Code. These override the values provided in the configuration file.
**python.venvPath**: Same as the **venvPath** setting described above. **python.venvPath**: Same as the **venvPath** setting described above.
**python.analysis.typeshedPaths**: An array of typeshed paths to search. Pyright supports only one such path. If provided in the VS Code setting, the first entry overrides the **typeshedPath** configuration file entry described above. **python.analysis.typeshedPaths**: An array of typeshed paths to search. Pyright supports only one such path. If provided in the VS Code setting, the first entry overrides the **typeshedPath** configuration file entry described above.
## Sample Config File ## Sample Config File
The following is an example of a pyright config file: The following is an example of a pyright config file:
```json ```json
{ {
"include": [ "include": ["src"],
"src"
],
"exclude": [
"**/node_modules",
"**/__pycache__",
"src/experimental",
"src/web/node_modules",
"src/typestubs"
],
"ignore": [ "exclude": ["**/node_modules", "**/__pycache__", "src/experimental", "src/web/node_modules", "src/typestubs"],
"src/oldstuff"
],
"typingsPath": "src/typestubs", "ignore": ["src/oldstuff"],
"venvPath": "/home/foo/.venvs",
"reportTypeshedErrors": false, "typingsPath": "src/typestubs",
"reportMissingImports": true, "venvPath": "/home/foo/.venvs",
"reportMissingTypeStubs": false,
"pythonVersion": "3.6", "reportTypeshedErrors": false,
"pythonPlatform": "Linux", "reportMissingImports": true,
"reportMissingTypeStubs": false,
"executionEnvironments": [ "pythonVersion": "3.6",
{ "pythonPlatform": "Linux",
"root": "src/web",
"pythonVersion": "3.5", "executionEnvironments": [
"pythonPlatform": "Windows", {
"extraPaths": [ "root": "src/web",
"src/service_libs" "pythonVersion": "3.5",
] "pythonPlatform": "Windows",
}, "extraPaths": ["src/service_libs"]
{ },
"root": "src/sdk", {
"pythonVersion": "3.0", "root": "src/sdk",
"extraPaths": [ "pythonVersion": "3.0",
"src/backend" "extraPaths": ["src/backend"],
], "venv": "venv_bar"
"venv": "venv_bar" },
}, {
{ "root": "src/tests",
"root": "src/tests", "extraPaths": ["src/tests/e2e", "src/sdk"]
"extraPaths": [ },
"src/tests/e2e", {
"src/sdk" "root": "src"
] }
}, ]
{
"root": "src"
}
]
} }
``` ```

8
docs/getting-started.md
View File

@ -3,12 +3,12 @@
A static type checker like pyright can add incremental value to your source code as more type information is provided. A static type checker like pyright can add incremental value to your source code as more type information is provided.
Here is a typical progression: Here is a typical progression:
1. Install pyright (either the VS Code extension or command-line tool). 1. Install pyright (either the VS Code extension or command-line tool).
2. Write a minimal `pyrightconfig.json` that defines `include` entries. Place the config file in your project’s top-level directory and commit it to your repo. 2. Write a minimal `pyrightconfig.json` that defines `include` entries. Place the config file in your project's top-level directory and commit it to your repo.
3. Run pyright over your source base with the default settings. Fix any errors and warnings that it emits. 3. Run pyright over your source base with the default settings. Fix any errors and warnings that it emits.
4. Enable the `reportMissingTypeStubs` setting in the config file and add (minimal) type stub files for the imported files. You may wish to create a “typestubs” directory within your code base -- a common location for all of your custom type stub files. You may be able to find preexisting type stub files for some of your imports within the typeshed repo (in the [third-party directory](https://github.com/python/typeshed/tree/master/third_party)). 4. Enable the `reportMissingTypeStubs` setting in the config file and add (minimal) type stub files for the imported files. You may wish to create a "typestubs" directory within your code base -- a common location for all of your custom type stub files. You may be able to find preexisting type stub files for some of your imports within the typeshed repo (in the [third-party directory](https://github.com/python/typeshed/tree/master/third_party)).
5. Check in your custom type stub files and configure pyright to run as part of your continuous integration (CI) environment to keep the project “type clean”. 5. Check in your custom type stub files and configure pyright to run as part of your continuous integration (CI) environment to keep the project "type clean".
6. Incrementally add type annotations to your code files. The annotations that provide most value are on function input parameters, instance variables, and return parameters (in that order). Note that annotation of variables (instance, class and local) requires Python 3.6 or newer. 6. Incrementally add type annotations to your code files. The annotations that provide most value are on function input parameters, instance variables, and return parameters (in that order). Note that annotation of variables (instance, class and local) requires Python 3.6 or newer.
7. Enable stricter type checking options like "reportOptionalSubscript", "reportOptionalMemberAccess", "reportOptionalCall", and "reportUntypedFunctionDecorator". 7. Enable stricter type checking options like "reportOptionalSubscript", "reportOptionalMemberAccess", "reportOptionalCall", and "reportUntypedFunctionDecorator".
8. On a file-by-file basis, enable all type checking options by adding the comment `# pyright: strict` somewhere in the file. 8. On a file-by-file basis, enable all type checking options by adding the comment `# pyright: strict` somewhere in the file.

7
docs/import-resolution.md
View File

@ -2,9 +2,8 @@
Pyright resolves external imports based on several configuration settings. If a venvPath and venv are specified, these are used to locate the `site-packages` directory within the virtual environment. Pyright resolves external imports based on several configuration settings. If a venvPath and venv are specified, these are used to locate the `site-packages` directory within the virtual environment.
If no venvPath is specified, Pyright falls back to the paths found in the default python interpreter’s search paths (or the python interpreter pointed to by the “python.pythonPath” setting in VS Code). Only directory-based paths are supported (as opposed to zip files or other loader packages). If no venvPath is specified, Pyright falls back to the paths found in the default python interpreter's search paths (or the python interpreter pointed to by the "python.pythonPath" setting in VS Code). Only directory-based paths are supported (as opposed to zip files or other loader packages).
The Pyright configuration file supports “execution environment” definitions, each of which can define additional paths. These are searched in addition to the venv or PYTHONPATH directories. The Pyright configuration file supports "execution environment" definitions, each of which can define additional paths. These are searched in addition to the venv or PYTHONPATH directories.
If Pyright is reporting import resolution errors, additional diagnostic information may help you determine why. If you are using the command-line version, try adding the “--verbose” switch. If you are using the VS Code extension, look at the “Output” window (View -> Output) and choose the “Pyright” view from the popup menu.
If Pyright is reporting import resolution errors, additional diagnostic information may help you determine why. If you are using the command-line version, try adding the "--verbose" switch. If you are using the VS Code extension, look at the "Output" window (View -> Output) and choose the "Pyright" view from the popup menu.

35
docs/internals.md
View File

@ -2,15 +2,14 @@
## Code Structure ## Code Structure
* client/src/extension.ts: Language Server Protocol (LSP) client entry point for VS Code extension. - client/src/extension.ts: Language Server Protocol (LSP) client entry point for VS Code extension.
* client/typeshed-fallback/: Recent copy of Typeshed type stub files for Python stdlib - client/typeshed-fallback/: Recent copy of Typeshed type stub files for Python stdlib
* server/src/pyright.ts: Main entry point for command-line tool - server/src/pyright.ts: Main entry point for command-line tool
* server/src/server.ts: Main entry point for LSP server - server/src/server.ts: Main entry point for LSP server
* server/src/analyzer: Modules that perform analysis passes over Python parse tree - server/src/analyzer: Modules that perform analysis passes over Python parse tree
* server/src/common: Modules that are common to the parser and analyzer - server/src/common: Modules that are common to the parser and analyzer
* server/src/parser: Modules that perform tokenization and parsing of Python source - server/src/parser: Modules that perform tokenization and parsing of Python source
* server/src/tests: Tests for the parser and analyzer - server/src/tests: Tests for the parser and analyzer
## Core Concepts ## Core Concepts
@ -22,12 +21,11 @@ The program tracks multiple [sourceFile](https://github.com/Microsoft/pyright/bl
The program makes use of an [importResolver](https://github.com/Microsoft/pyright/blob/master/server/src/analyzer/importResolver.ts) to resolve the imported modules referenced within each source file. The program makes use of an [importResolver](https://github.com/Microsoft/pyright/blob/master/server/src/analyzer/importResolver.ts) to resolve the imported modules referenced within each source file.
## Analysis Phases ## Analysis Phases
Pyright performs the following analysis phases for each source file. Pyright performs the following analysis phases for each source file.
The [tokenizer](https://github.com/Microsoft/pyright/blob/master/server/src/parser/tokenizer.ts) is responsible for converting the file’s string contents into a stream of tokens. White space, comments, and some end-of-line characters are ignored, as they are not needed by the parser. The [tokenizer](https://github.com/Microsoft/pyright/blob/master/server/src/parser/tokenizer.ts) is responsible for converting the file's string contents into a stream of tokens. White space, comments, and some end-of-line characters are ignored, as they are not needed by the parser.
The [parser](https://github.com/Microsoft/pyright/blob/master/server/src/parser/parser.ts) is responsible for converting the token stream into a parse tree. A generalized [parseTreeWalker](https://github.com/Microsoft/pyright/blob/master/server/src/analyzer/parseTreeWalker.ts) provides a convenient way to traverse the parse tree. All subsequent analysis phases utilize the parseTreeWalker. The [parser](https://github.com/Microsoft/pyright/blob/master/server/src/parser/parser.ts) is responsible for converting the token stream into a parse tree. A generalized [parseTreeWalker](https://github.com/Microsoft/pyright/blob/master/server/src/analyzer/parseTreeWalker.ts) provides a convenient way to traverse the parse tree. All subsequent analysis phases utilize the parseTreeWalker.
@ -37,11 +35,12 @@ The [checker](https://github.com/Microsoft/pyright/blob/master/server/src/analyz
## Type Checking Concepts ## Type Checking Concepts
Pyright uses an internal type called “Unknown” to represent types that are not annotated and cannot be inferred. Unknown is generally treated like the “Any” type in terms of type checking, but it provides a way for developers to know when type annotations are missing and could provide additional value. Pyright uses an internal type called "Unknown" to represent types that are not annotated and cannot be inferred. Unknown is generally treated like the "Any" type in terms of type checking, but it provides a way for developers to know when type annotations are missing and could provide additional value.
Pyright attempts to infer the types of global (module-level) variables, class variables, instance variables, and local variables. Return and yield types are also inferred. If type annotations are provided in these cases, the type annotation overrides any inferred types. Pyright attempts to infer the types of global (module-level) variables, class variables, instance variables, and local variables. Return and yield types are also inferred. If type annotations are provided in these cases, the type annotation overrides any inferred types.
Pyright supports type constraints (sometimes called “path constraints” or "type guards") to track assumptions that apply within certain code flow paths. For example, consider the following code: Pyright supports type constraints (sometimes called "path constraints" or "type guards") to track assumptions that apply within certain code flow paths. For example, consider the following code:
```python ```python
def (a: Optional[Union[str, List[str]]): def (a: Optional[Union[str, List[str]]):
if isinstance(a, str): if isinstance(a, str):
@ -55,6 +54,7 @@ def (a: Optional[Union[str, List[str]]):
In this example, the type evaluator knows that parameter a is either None, str, or List[str]. Within the first `if` clause, a is constrained to be a str. Within the `elif` clause, it is constrained to be a List[str], and within the `else` clause, it has to be None (by process of elimination). The type checker would therefore flag the final line as an error if the log method could not accept None as a parameter. In this example, the type evaluator knows that parameter a is either None, str, or List[str]. Within the first `if` clause, a is constrained to be a str. Within the `elif` clause, it is constrained to be a List[str], and within the `else` clause, it has to be None (by process of elimination). The type checker would therefore flag the final line as an error if the log method could not accept None as a parameter.
If the type constraint logic exhausts all possible subtypes, it can be assumed that a code path will never be taken. For example, consider the following: If the type constraint logic exhausts all possible subtypes, it can be assumed that a code path will never be taken. For example, consider the following:
```python ```python
def (a: Union[Foo, Bar]): def (a: Union[Foo, Bar]):
if isinstance(a, Foo): if isinstance(a, Foo):
@ -68,15 +68,16 @@ def (a: Union[Foo, Bar]):
a.do_something_3() a.do_something_3()
``` ```
In this case, the type of parameter “a” is initially “Union[Foo, Bar]”. Within the “if” clause, the type constraint logic will conclude that it must be of type “Foo”. Within the “elif” clause, it must be of type “Bar”. What type is it within the “else” clause? The type constraint system has eliminated all possible subtypes, so it gives it the type “Never”. This is generally indicates that there’s a logic error in the code because there’s way that code block will ever be executed. In this case, the type of parameter "a" is initially "Union[Foo, Bar]". Within the "if" clause, the type constraint logic will conclude that it must be of type "Foo". Within the "elif" clause, it must be of type "Bar". What type is it within the "else" clause? The type constraint system has eliminated all possible subtypes, so it gives it the type "Never". This is generally indicates that there's a logic error in the code because there's way that code block will ever be executed.
## Type Inference ## Type Inference
In cases where explicit type annotations are not provided, Pyright attempts to infer the types. The inferred return type of a function is determined from all of the return (and yield) statements within the function’s definition. The inferred type of a local variable is determined by the expression that is assigned to that variable. Likewise, the type of a member variable is inferred from all assignments to that member variable within its defining class. In cases where explicit type annotations are not provided, Pyright attempts to infer the types. The inferred return type of a function is determined from all of the return (and yield) statements within the function's definition. The inferred type of a local variable is determined by the expression that is assigned to that variable. Likewise, the type of a member variable is inferred from all assignments to that member variable within its defining class.
The types of input parameters cannot be inferred, with the exception of the “self” or “cls” parameter for instance members and class members, respectively. The types of input parameters cannot be inferred, with the exception of the "self" or "cls" parameter for instance members and class members, respectively.
If an inferred return type is unknown or partially unknown because input parameter types are not annotated, Pyright may still be able to infer the return type based on the types of arguments at the call site.
If an inferred return type is unknown or partially unknown because input parameter types are not annotated, Pyright may still be able to infer the return type based on the types of arguments at the call site.
``` ```
def add_values(a, b): def add_values(a, b):
return a + b return a + b

4
docs/settings.md
View File

@ -2,7 +2,7 @@
The Pyright VS Code extension honors the following settings. The Pyright VS Code extension honors the following settings.
**pyright.disableLanguageServices** [boolean]: Disables all language services except for “hover”. This includes type completion, signature completion, find definition, find references, and find symbols in file. This option is useful if you want to use pyright only as a type checker but want to run another Python language server for langue service features. **pyright.disableLanguageServices** [boolean]: Disables all language services except for "hover". This includes type completion, signature completion, find definition, find references, and find symbols in file. This option is useful if you want to use pyright only as a type checker but want to run another Python language server for langue service features.
**pyright.openFilesOnly** [boolean]: Determines whether pyright analyzes (and reports errors for) all files in the workspace, as indicated by the config file. If this option is set to true, pyright analyzes only open files. **pyright.openFilesOnly** [boolean]: Determines whether pyright analyzes (and reports errors for) all files in the workspace, as indicated by the config file. If this option is set to true, pyright analyzes only open files.
@ -13,5 +13,3 @@ The Pyright VS Code extension honors the following settings.
**python.pythonPath** [path]: Path to Python interpreter. **python.pythonPath** [path]: Path to Python interpreter.
**python.venvPath** [path]: Path to folder with subdirectories that contain virtual environments. **python.venvPath** [path]: Path to folder with subdirectories that contain virtual environments.

27
docs/type-stubs.md
View File

@ -1,28 +1,30 @@
# Type Stub Files # Type Stub Files
Type stubs are “.pyi” files that specify the public interface for a library. They use a variant of the Python syntax that allows for “...” to be used in place of any implementation details. Type stubs define the public contract for the library. Type stubs are ".pyi" files that specify the public interface for a library. They use a variant of the Python syntax that allows for "..." to be used in place of any implementation details. Type stubs define the public contract for the library.
## Importance of Type Stub Files ## Importance of Type Stub Files
Regardless of the search path, Pyright always attempts to resolve an import with a type stub (“.pyi”) file before falling back to a python source (“.py”) file. If a type stub cannot be located for an external import, that import will be treated as a “black box” for purposes of type analysis. Any symbols imported from these modules will be of type “Unknown”, and wildcard imports (of the form `from foo import *`) will not populate the module’s namespace with specific symbol names. Regardless of the search path, Pyright always attempts to resolve an import with a type stub (".pyi") file before falling back to a python source (".py") file. If a type stub cannot be located for an external import, that import will be treated as a "black box" for purposes of type analysis. Any symbols imported from these modules will be of type "Unknown", and wildcard imports (of the form `from foo import *`) will not populate the module's namespace with specific symbol names.
Why does Pyright not attempt (by default) to determine types from imported python sources? There are several reasons. Why does Pyright not attempt (by default) to determine types from imported python sources? There are several reasons.
1. Imported libraries can be quite large, so analyzing them can require significant time and computation. 1. Imported libraries can be quite large, so analyzing them can require significant time and computation.
2. Some libraries are thin shims on top of native (C++) libraries. Little or no type information would be inferable in these cases. 2. Some libraries are thin shims on top of native (C++) libraries. Little or no type information would be inferable in these cases.
3. Some libraries override Python’s default loader logic. Static analysis is not possible in these cases. 3. Some libraries override Python's default loader logic. Static analysis is not possible in these cases.
4. Type information inferred from source files is often of low value because many types cannot be inferred correctly. Even if concrete types can be inferred, generic type definitions cannot. 4. Type information inferred from source files is often of low value because many types cannot be inferred correctly. Even if concrete types can be inferred, generic type definitions cannot.
5. Type analysis would expose all symbols from an imported module, even those that are not meant to be exposed by the author. Unlike many other languages, Python offers no way of differentiating between a symbol that is meant to be exported and one that isn’t. 5. Type analysis would expose all symbols from an imported module, even those that are not meant to be exposed by the author. Unlike many other languages, Python offers no way of differentiating between a symbol that is meant to be exported and one that isn't.
If you’re serious about static type checking for your Python source base, it’s highly recommended that you use type stub files for all external imports. If you are unable to find a type stub for a particular library, the recommended approach is to create a custom type stub file that defines the portion of that module’s interface used by your code. More library authors have started to provide type stub files. If you're serious about static type checking for your Python source base, it's highly recommended that you use type stub files for all external imports. If you are unable to find a type stub for a particular library, the recommended approach is to create a custom type stub file that defines the portion of that module's interface used by your code. More library authors have started to provide type stub files.
## Generating Type Stubs ## Generating Type Stubs
If you use only a few classes, methods or functions within a library, writing a type stub file by hand is feasible. For large libraries, this can become tedious and error-prone. Pyright can generate “draft” versions of type stub files for you.
To generate a type stub file from within VS Code, enable the reportMissingTypeStubs” setting in your pyrightconfig.json file or by adding a comment `# pyright: reportMissingTypeStubs=true` to individual source files. Make sure you have the target library installed in the python environment that pyright is configured to use for import resolution. Optionally specify a “typingsPath” in your pyrightconfig.json file. This is where pyright will generate your type stub files. By default, the typingsPath is set to "./typings". If you use only a few classes, methods or functions within a library, writing a type stub file by hand is feasible. For large libraries, this can become tedious and error-prone. Pyright can generate "draft" versions of type stub files for you.
To generate a type stub file from within VS Code, enable the reportMissingTypeStubs" setting in your pyrightconfig.json file or by adding a comment `# pyright: reportMissingTypeStubs=true` to individual source files. Make sure you have the target library installed in the python environment that pyright is configured to use for import resolution. Optionally specify a "typingsPath" in your pyrightconfig.json file. This is where pyright will generate your type stub files. By default, the typingsPath is set to "./typings".
### Generating Type Stubs in VS Code ### Generating Type Stubs in VS Code
If “reportMissingTypeStubs” is enabled, pyright will highlight any imports that have no type stub. Hover over the error message, and you will see a “Quick Fix” link. Clicking on this link will reveal a popup menu item titled “Create Type Stub For XXX”. The example below shows a missing typestub for the `django` library.
If "reportMissingTypeStubs" is enabled, pyright will highlight any imports that have no type stub. Hover over the error message, and you will see a "Quick Fix" link. Clicking on this link will reveal a popup menu item titled "Create Type Stub For XXX". The example below shows a missing typestub for the `django` library.
![Pyright](/docs/img/CreateTypeStub1.png) ![Pyright](/docs/img/CreateTypeStub1.png)
@ -31,21 +33,23 @@ Click on the menu item to create the type stub. Depending on the size of the lib
![Pyright](/docs/img/CreateTypeStub2.png) ![Pyright](/docs/img/CreateTypeStub2.png)
### Generating Type Stubs from Command Line ### Generating Type Stubs from Command Line
The command-line version of pyright can also be used to generate type stubs. As with the VS Code version, it must be run within the context of your configured project. Then type `pyright --createstub [import-name]`. The command-line version of pyright can also be used to generate type stubs. As with the VS Code version, it must be run within the context of your configured project. Then type `pyright --createstub [import-name]`.
For example: For example:
`pyright --createstub django` `pyright --createstub django`
### Cleaning Up Generated Type Stubs ### Cleaning Up Generated Type Stubs
Pyright can give you a head start by creating type stubs, but you will typically need to clean up the first draft, fixing various errors and omissions that pyright was not able to infer from the original library code. Pyright can give you a head start by creating type stubs, but you will typically need to clean up the first draft, fixing various errors and omissions that pyright was not able to infer from the original library code.
A few common situations that need to be cleaned up: A few common situations that need to be cleaned up:
1. When generating a “.pyi” file, pyright removes any imports that are not referenced. Sometimes libraries import symbols that are meant to be simply re-exported from a module even though they are not referenced internally to that module. In such cases, you will need to manually add back these imports. Pyright does not perform this import culling in `__init__.pyi` files because this re-export technique is especially common in such files. 1. When generating a ".pyi" file, pyright removes any imports that are not referenced. Sometimes libraries import symbols that are meant to be simply re-exported from a module even though they are not referenced internally to that module. In such cases, you will need to manually add back these imports. Pyright does not perform this import culling in `__init__.pyi` files because this re-export technique is especially common in such files.
2. Some libraries attempt to import modules within a try statement. These constructs don’t work well in type stub files because they cannot be evaluated statically. Pyright omits any try statements when creating “.pyi” files, so you may need to add back in these import statements. 2. Some libraries attempt to import modules within a try statement. These constructs don't work well in type stub files because they cannot be evaluated statically. Pyright omits any try statements when creating ".pyi" files, so you may need to add back in these import statements.
3. Decorator functions are especially problematic for static type analyzers. Unless properly typed, they completely hide the signature of any class or function they are applied to. For this reason, it is highly recommended that you enable the “reportUntypedFunctionDecorator” and “reportUntypedClassDecorator” switches in pyrightconfig.json. Most decorators simply return the same function they are passed. Those can easily be annotated with a TypeVar like this: 3. Decorator functions are especially problematic for static type analyzers. Unless properly typed, they completely hide the signature of any class or function they are applied to. For this reason, it is highly recommended that you enable the "reportUntypedFunctionDecorator" and "reportUntypedClassDecorator" switches in pyrightconfig.json. Most decorators simply return the same function they are passed. Those can easily be annotated with a TypeVar like this:
``` ```
from typings import Any, Callable, TypeVar from typings import Any, Callable, TypeVar
@ -54,4 +58,3 @@ _FuncT = TypeVar('_FuncT', bound=Callable[..., Any])
def my_decorator(*args, **kw) -> Callable[[_FuncT], _FuncT]: ... def my_decorator(*args, **kw) -> Callable[[_FuncT], _FuncT]: ...
``` ```

2
index.debug.js
View File

@ -3,4 +3,4 @@
// Stash the base directory into a global variable. // Stash the base directory into a global variable.
global.__rootDirectory = __dirname + '/dist/'; global.__rootDirectory = __dirname + '/dist/';
require('./client/server/pyright') require('./client/server/pyright');

2
index.js
View File

@ -3,4 +3,4 @@
// Stash the base directory into a global variable. // Stash the base directory into a global variable.
global.__rootDirectory = __dirname + '/dist/'; global.__rootDirectory = __dirname + '/dist/';
require('./dist/pyright') require('./dist/pyright');

85
package-lock.json generated
View File

@ -74,9 +74,9 @@
} }
}, },
"@types/json-schema": { "@types/json-schema": {
"version": "7.0.3", "version": "7.0.4",
"resolved": "https://registry.npmjs.org/@types/json-schema/-/json-schema-7.0.3.tgz", "resolved": "https://registry.npmjs.org/@types/json-schema/-/json-schema-7.0.4.tgz",
"integrity": "sha512-Il2DtDVRGDcqjDtE+rF8iqg1CArehSK84HZJCT7AMITlyXRBpuPhqGLDQMowraqqu1coEaimg4ZOqggt6L6L+A==", "integrity": "sha512-8+KAKzEvSUdeo+kmqnKrqgeE+LcA0tjYWFY7RPProVYwnqDjukzO+3b6dLD56rYX5TdWejnEOLJYOIeh4CXKuA==",
"dev": true "dev": true
}, },
"@types/minimatch": { "@types/minimatch": {
@ -85,25 +85,19 @@
"integrity": "sha512-tHq6qdbT9U1IRSGf14CL0pUlULksvY9OZ+5eEgl1N7t+OA3tGvNpxJCzuKQlsNgCVwbAs670L1vcVQi8j9HjnA==", "integrity": "sha512-tHq6qdbT9U1IRSGf14CL0pUlULksvY9OZ+5eEgl1N7t+OA3tGvNpxJCzuKQlsNgCVwbAs670L1vcVQi8j9HjnA==",
"dev": true "dev": true
}, },
"@types/mocha": {
"version": "5.2.7",
"resolved": "https://registry.npmjs.org/@types/mocha/-/mocha-5.2.7.tgz",
"integrity": "sha512-NYrtPht0wGzhwe9+/idPaBB+TqkY9AhTvOLMkThm0IoEfLaiVQZwBwyJ5puCkO3AUCWrmcoePjp2mbFocKy4SQ==",
"dev": true
},
"@types/node": { "@types/node": {
"version": "12.12.21", "version": "12.12.28",
"resolved": "https://registry.npmjs.org/@types/node/-/node-12.12.21.tgz", "resolved": "https://registry.npmjs.org/@types/node/-/node-12.12.28.tgz",
"integrity": "sha512-8sRGhbpU+ck1n0PGAUgVrWrWdjSW2aqNeyC15W88GRsMpSwzv6RJGlLhE7s2RhVSOdyDmxbqlWSeThq4/7xqlA==", "integrity": "sha512-g73GJYJDXgf0jqg+P9S8h2acWbDXNkoCX8DLtJVu7Fkn788pzQ/oJsrdJz/2JejRf/SjfZaAhsw+3nd1D5EWGg==",
"dev": true "dev": true
}, },
"@typescript-eslint/eslint-plugin": { "@typescript-eslint/eslint-plugin": {
"version": "2.12.0", "version": "2.21.0",
"resolved": "https://registry.npmjs.org/@typescript-eslint/eslint-plugin/-/eslint-plugin-2.12.0.tgz", "resolved": "https://registry.npmjs.org/@typescript-eslint/eslint-plugin/-/eslint-plugin-2.21.0.tgz",
"integrity": "sha512-1t4r9rpLuEwl3hgt90jY18wJHSyb0E3orVL3DaqwmpiSDHmHiSspVsvsFF78BJ/3NNG3qmeso836jpuBWYziAA==", "integrity": "sha512-b5jjjDMxzcjh/Sbjuo7WyhrQmVJg0WipTHQgXh5Xwx10uYm6nPWqN1WGOsaNq4HR3Zh4wUx4IRQdDkCHwyewyw==",
"dev": true, "dev": true,
"requires": { "requires": {
"@typescript-eslint/experimental-utils": "2.12.0", "@typescript-eslint/experimental-utils": "2.21.0",
"eslint-utils": "^1.4.3", "eslint-utils": "^1.4.3",
"functional-red-black-tree": "^1.0.1", "functional-red-black-tree": "^1.0.1",
"regexpp": "^3.0.0", "regexpp": "^3.0.0",
@ -111,39 +105,39 @@
} }
}, },
"@typescript-eslint/experimental-utils": { "@typescript-eslint/experimental-utils": {
"version": "2.12.0", "version": "2.21.0",
"resolved": "https://registry.npmjs.org/@typescript-eslint/experimental-utils/-/experimental-utils-2.12.0.tgz", "resolved": "https://registry.npmjs.org/@typescript-eslint/experimental-utils/-/experimental-utils-2.21.0.tgz",
"integrity": "sha512-jv4gYpw5N5BrWF3ntROvCuLe1IjRenLy5+U57J24NbPGwZFAjhnM45qpq0nDH1y/AZMb3Br25YiNVwyPbz6RkA==", "integrity": "sha512-olKw9JP/XUkav4lq0I7S1mhGgONJF9rHNhKFn9wJlpfRVjNo3PPjSvybxEldvCXnvD+WAshSzqH5cEjPp9CsBA==",
"dev": true, "dev": true,
"requires": { "requires": {
"@types/json-schema": "^7.0.3", "@types/json-schema": "^7.0.3",
"@typescript-eslint/typescript-estree": "2.12.0", "@typescript-eslint/typescript-estree": "2.21.0",
"eslint-scope": "^5.0.0" "eslint-scope": "^5.0.0"
} }
}, },
"@typescript-eslint/parser": { "@typescript-eslint/parser": {
"version": "2.12.0", "version": "2.21.0",
"resolved": "https://registry.npmjs.org/@typescript-eslint/parser/-/parser-2.12.0.tgz", "resolved": "https://registry.npmjs.org/@typescript-eslint/parser/-/parser-2.21.0.tgz",
"integrity": "sha512-lPdkwpdzxEfjI8TyTzZqPatkrswLSVu4bqUgnB03fHSOwpC7KSerPgJRgIAf11UGNf7HKjJV6oaPZI4AghLU6g==", "integrity": "sha512-VrmbdrrrvvI6cPPOG7uOgGUFXNYTiSbnRq8ZMyuGa4+qmXJXVLEEz78hKuqupvkpwJQNk1Ucz1TenrRP90gmBg==",
"dev": true, "dev": true,
"requires": { "requires": {
"@types/eslint-visitor-keys": "^1.0.0", "@types/eslint-visitor-keys": "^1.0.0",
"@typescript-eslint/experimental-utils": "2.12.0", "@typescript-eslint/experimental-utils": "2.21.0",
"@typescript-eslint/typescript-estree": "2.12.0", "@typescript-eslint/typescript-estree": "2.21.0",
"eslint-visitor-keys": "^1.1.0" "eslint-visitor-keys": "^1.1.0"
} }
}, },
"@typescript-eslint/typescript-estree": { "@typescript-eslint/typescript-estree": {
"version": "2.12.0", "version": "2.21.0",
"resolved": "https://registry.npmjs.org/@typescript-eslint/typescript-estree/-/typescript-estree-2.12.0.tgz", "resolved": "https://registry.npmjs.org/@typescript-eslint/typescript-estree/-/typescript-estree-2.21.0.tgz",
"integrity": "sha512-rGehVfjHEn8Frh9UW02ZZIfJs6SIIxIu/K1bbci8rFfDE/1lQ8krIJy5OXOV3DVnNdDPtoiPOdEANkLMrwXbiQ==", "integrity": "sha512-NC/nogZNb9IK2MEFQqyDBAciOT8Lp8O3KgAfvHx2Skx6WBo+KmDqlU3R9KxHONaijfTIKtojRe3SZQyMjr3wBw==",
"dev": true, "dev": true,
"requires": { "requires": {
"debug": "^4.1.1", "debug": "^4.1.1",
"eslint-visitor-keys": "^1.1.0", "eslint-visitor-keys": "^1.1.0",
"glob": "^7.1.6", "glob": "^7.1.6",
"is-glob": "^4.0.1", "is-glob": "^4.0.1",
"lodash.unescape": "4.0.1", "lodash": "^4.17.15",
"semver": "^6.3.0", "semver": "^6.3.0",
"tsutils": "^3.17.1" "tsutils": "^3.17.1"
} }
@ -544,6 +538,15 @@
} }
} }
}, },
"eslint-config-prettier": {
"version": "6.10.0",
"resolved": "https://registry.npmjs.org/eslint-config-prettier/-/eslint-config-prettier-6.10.0.tgz",
"integrity": "sha512-AtndijGte1rPILInUdHjvKEGbIV06NuvPrqlIEaEaWtbtvJh464mDeyGMdZEQMsGvC0ZVkiex1fSNcC4HAbRGg==",
"dev": true,
"requires": {
"get-stdin": "^6.0.0"
}
},
"eslint-plugin-simple-import-sort": { "eslint-plugin-simple-import-sort": {
"version": "5.0.1", "version": "5.0.1",
"resolved": "https://registry.npmjs.org/eslint-plugin-simple-import-sort/-/eslint-plugin-simple-import-sort-5.0.1.tgz", "resolved": "https://registry.npmjs.org/eslint-plugin-simple-import-sort/-/eslint-plugin-simple-import-sort-5.0.1.tgz",
@ -738,6 +741,12 @@
"integrity": "sha1-GwqzvVU7Kg1jmdKcDj6gslIHgyc=", "integrity": "sha1-GwqzvVU7Kg1jmdKcDj6gslIHgyc=",
"dev": true "dev": true
}, },
"get-stdin": {
"version": "6.0.0",
"resolved": "https://registry.npmjs.org/get-stdin/-/get-stdin-6.0.0.tgz",
"integrity": "sha512-jp4tHawyV7+fkkSKyvjuLZswblUtz+SQKzSWnBbii16BuZksJlU1wuBYXY75r+duh/llF1ur6oNwi+2ZzjKZ7g==",
"dev": true
},
"glob": { "glob": {
"version": "7.1.6", "version": "7.1.6",
"resolved": "https://registry.npmjs.org/glob/-/glob-7.1.6.tgz", "resolved": "https://registry.npmjs.org/glob/-/glob-7.1.6.tgz",
@ -1021,12 +1030,6 @@
"integrity": "sha512-8xOcRHvCjnocdS5cpwXQXVzmmh5e5+saE2QGoeQmbKmRS6J3VQppPOIt0MnmE+4xlZoumy0GPG0D0MVIQbNA1A==", "integrity": "sha512-8xOcRHvCjnocdS5cpwXQXVzmmh5e5+saE2QGoeQmbKmRS6J3VQppPOIt0MnmE+4xlZoumy0GPG0D0MVIQbNA1A==",
"dev": true "dev": true
}, },
"lodash.unescape": {
"version": "4.0.1",
"resolved": "https://registry.npmjs.org/lodash.unescape/-/lodash.unescape-4.0.1.tgz",
"integrity": "sha1-vyJJiGzlFM2hEvrpIYzcBlIR/Jw=",
"dev": true
},
"loud-rejection": { "loud-rejection": {
"version": "1.6.0", "version": "1.6.0",
"resolved": "https://registry.npmjs.org/loud-rejection/-/loud-rejection-1.6.0.tgz", "resolved": "https://registry.npmjs.org/loud-rejection/-/loud-rejection-1.6.0.tgz",
@ -1298,6 +1301,12 @@
"integrity": "sha1-IZMqVJ9eUv/ZqCf1cOBL5iqX2lQ=", "integrity": "sha1-IZMqVJ9eUv/ZqCf1cOBL5iqX2lQ=",
"dev": true "dev": true
}, },
"prettier": {
"version": "1.19.1",
"resolved": "https://registry.npmjs.org/prettier/-/prettier-1.19.1.tgz",
"integrity": "sha512-s7PoyDv/II1ObgQunCbB9PdLmUcBZcnWOcxDh7O0N/UwDEsHyqkW+Qh28jW+mVuCdx7gLB0BotYI1Y6uI9iyew==",
"dev": true
},
"progress": { "progress": {
"version": "2.0.3", "version": "2.0.3",
"resolved": "https://registry.npmjs.org/progress/-/progress-2.0.3.tgz", "resolved": "https://registry.npmjs.org/progress/-/progress-2.0.3.tgz",
@ -1702,9 +1711,9 @@
"dev": true "dev": true
}, },
"typescript": { "typescript": {
"version": "3.7.3", "version": "3.8.2",
"resolved": "https://registry.npmjs.org/typescript/-/typescript-3.7.3.tgz", "resolved": "https://registry.npmjs.org/typescript/-/typescript-3.8.2.tgz",
"integrity": "sha512-Mcr/Qk7hXqFBXMN7p7Lusj1ktCBydylfQM/FZCk5glCNQJrCUKPkMHdo9R0MTFWsC/4kPFvDS0fDPvukfCkFsw==", "integrity": "sha512-EgOVgL/4xfVrCMbhYKUQTdF37SQn4Iw73H5BgCrF1Abdun7Kwy/QZsE/ssAy0y4LxBbvua3PIbFsbRczWWnDdQ==",
"dev": true "dev": true
}, },
"uri-js": { "uri-js": {

11
package.json
View File

@ -24,14 +24,15 @@
"package": "npm run install:all && npm run clean && npm run build:serverProd && npm run build:cli && cd client && npx vsce package && cd .." "package": "npm run install:all && npm run clean && npm run build:serverProd && npm run build:cli && cd client && npx vsce package && cd .."
}, },
"devDependencies": { "devDependencies": {
"@types/mocha": "^5.2.7", "@types/node": "^12.12.28",
"@types/node": "^12.12.21", "@typescript-eslint/eslint-plugin": "^2.21.0",
"@typescript-eslint/eslint-plugin": "^2.12.0", "@typescript-eslint/parser": "^2.21.0",
"@typescript-eslint/parser": "^2.12.0",
"del-cli": "^3.0.0", "del-cli": "^3.0.0",
"eslint": "^6.8.0", "eslint": "^6.8.0",
"eslint-config-prettier": "^6.10.0",
"eslint-plugin-simple-import-sort": "^5.0.1", "eslint-plugin-simple-import-sort": "^5.0.1",
"typescript": "^3.7.3" "prettier": "1.19.1",
"typescript": "^3.8.2"
}, },
"main": "index.js", "main": "index.js",
"bin": { "bin": {

3
server/.eslintignore
View File

@ -1 +1,2 @@
**/tests/fourslash/** **/client/server/**
**/tests/fourslash/**

4
server/.vscode/settings.json vendored
View File

@ -1,4 +0,0 @@
{
"javascript.format.insertSpaceAfterOpeningAndBeforeClosingTemplateStringBraces": true,
"typescript.format.insertSpaceAfterOpeningAndBeforeClosingTemplateStringBraces": true
}

2
server/copyTypeshedFallback.js
View File

@ -1,4 +1,5 @@
#!/usr/bin/env node #!/usr/bin/env node
/* eslint-disable @typescript-eslint/no-var-requires */
// This script helps build the command-line version of pyright // This script helps build the command-line version of pyright
// by copying the typeshed-fallback directory to the dist directory. // by copying the typeshed-fallback directory to the dist directory.
@ -10,4 +11,3 @@ fsExtra.emptyDirSync('../dist');
fsExtra.mkdirSync('../dist/typeshed-fallback'); fsExtra.mkdirSync('../dist/typeshed-fallback');
fsExtra.copySync('../client/typeshed-fallback', '../dist/typeshed-fallback'); fsExtra.copySync('../client/typeshed-fallback', '../dist/typeshed-fallback');

89
server/customInstallServerIntoExtension.js
View File

@ -1,75 +1,88 @@
#!/usr/bin/env node #!/usr/bin/env node
/* eslint-disable @typescript-eslint/no-var-requires */
// This file is based on the "installServerIntoExtension" that ships with the // This file is based on the "installServerIntoExtension" that ships with the
// vscode-languagserver node package. We needed to modify it because the original // vscode-languagserver node package. We needed to modify it because the original
// version does not copy the package-lock.json file, and it uses npm update // version does not copy the package-lock.json file, and it uses npm update
// rather than npm install. // rather than npm install.
var path = require('path'); const path = require('path');
var fs = require('fs'); const fs = require('fs');
var cp = require('child_process'); const cp = require('child_process');
var extensionDirectory = process.argv[2]; let extensionDirectory = process.argv[2];
if (!extensionDirectory) { if (!extensionDirectory) {
console.error('No extension directory provided.'); console.error('No extension directory provided.');
process.exit(1) process.exit(1);
} }
extensionDirectory = path.resolve(extensionDirectory) extensionDirectory = path.resolve(extensionDirectory);
if (!fs.existsSync(extensionDirectory)) { if (!fs.existsSync(extensionDirectory)) {
console.error('Extension directory ' + extensionDirectory + ' doesn\'t exist on disk.'); console.error('Extension directory ' + extensionDirectory + " doesn't exist on disk.");
process.exit(1); process.exit(1);
} }
var packageFile = process.argv[3]; let packageFile = process.argv[3];
if (!packageFile) { if (!packageFile) {
console.error('No package.json file provided.'); console.error('No package.json file provided.');
process.exit(1); process.exit(1);
} }
packageFile = path.resolve(packageFile); packageFile = path.resolve(packageFile);
if (!fs.existsSync(packageFile)) { if (!fs.existsSync(packageFile)) {
console.error('Package file ' + packageFile + ' doesn\'t exist on disk.'); console.error('Package file ' + packageFile + " doesn't exist on disk.");
process.exit(1); process.exit(1);
} }
var tsconfigFile = process.argv[4]; let tsconfigFile = process.argv[4];
if (!tsconfigFile) { if (!tsconfigFile) {
console.error('No tsconfig.json file provided'); console.error('No tsconfig.json file provided');
process.exit(1); process.exit(1);
} }
tsconfigFile = path.resolve(tsconfigFile); tsconfigFile = path.resolve(tsconfigFile);
if (!fs.existsSync(tsconfigFile)) { if (!fs.existsSync(tsconfigFile)) {
console.error('tsconfig file ' + tsconfigFile + ' doesn\'t exist on disk.') console.error('tsconfig file ' + tsconfigFile + " doesn't exist on disk.");
process.exit(1); process.exit(1);
} }
var extensionServerDirectory = path.join(extensionDirectory, 'server') const extensionServerDirectory = path.join(extensionDirectory, 'server');
var json = require(tsconfigFile); const json = require(tsconfigFile);
var compilerOptions = json.compilerOptions; const compilerOptions = json.compilerOptions;
if (compilerOptions) { if (compilerOptions) {
var outDir = compilerOptions.outDir; const outDir = compilerOptions.outDir;
if (!outDir || path.join(path.dirname(tsconfigFile), outDir) !== extensionServerDirectory) { if (!outDir || path.join(path.dirname(tsconfigFile), outDir) !== extensionServerDirectory) {
console.error('outDir in ' + process.argv[4] + ' must point to ' + extensionServerDirectory + ' but it points to ' + path.join(path.dirname(tsconfigFile), outDir)); console.error(
console.error('Please change outDir in ' + process.argv[4] + ' to ' + path.relative(path.dirname(tsconfigFile), extensionServerDirectory).replace(/\\/g, '/')); 'outDir in ' +
process.exit(1); process.argv[4] +
} ' must point to ' +
extensionServerDirectory +
' but it points to ' +
path.join(path.dirname(tsconfigFile), outDir)
);
console.error(
'Please change outDir in ' +
process.argv[4] +
' to ' +
path.relative(path.dirname(tsconfigFile), extensionServerDirectory).replace(/\\/g, '/')
);
process.exit(1);
}
} }
if (!fs.existsSync(extensionServerDirectory)) { if (!fs.existsSync(extensionServerDirectory)) {
fs.mkdirSync(extensionServerDirectory); fs.mkdirSync(extensionServerDirectory);
} }
var dest = path.join(extensionServerDirectory, 'package.json'); const dest = path.join(extensionServerDirectory, 'package.json');
console.log('Copying package.json to extension\'s server location...'); console.log("Copying package.json to extension's server location...");
fs.writeFileSync(dest, fs.readFileSync(packageFile)); fs.writeFileSync(dest, fs.readFileSync(packageFile));
var packageLockFile = process.argv[5]; let packageLockFile = process.argv[5];
if (fs.existsSync(packageLockFile)) { if (fs.existsSync(packageLockFile)) {
const packageLockFileDest = path.join(extensionServerDirectory, 'package-lock.json'); const packageLockFileDest = path.join(extensionServerDirectory, 'package-lock.json');
packageLockFile = path.resolve(packageLockFile); packageLockFile = path.resolve(packageLockFile);
console.log('Copying package-lock.json to extension\'s server location...'); console.log("Copying package-lock.json to extension's server location...");
fs.writeFileSync(packageLockFileDest, fs.readFileSync(packageLockFile)); fs.writeFileSync(packageLockFileDest, fs.readFileSync(packageLockFile));
} }
console.log('Installing server npm modules into extension\'s server location...'); console.log("Installing server npm modules into extension's server location...");
process.chdir(extensionServerDirectory) process.chdir(extensionServerDirectory);
cp.execSync('npm install --production --prefix'); cp.execSync('npm install --production --prefix');

48
server/jest.config.js
View File

@ -1,33 +1,25 @@
/* /*
* jest.config.js * jest.config.js
* *
* Configuration for jest tests. * Configuration for jest tests.
*/ */
module.exports = { module.exports = {
roots: [ roots: ['<rootDir>/src/tests'],
'<rootDir>/src/tests' transform: {
], '^.+\\.tsx?$': 'ts-jest'
transform: { },
'^.+\\.tsx?$': 'ts-jest' testRegex: '(/__tests__/.*|(\\.|/)(test|spec))\\.tsx?$',
}, moduleFileExtensions: ['ts', 'tsx', 'js', 'jsx'],
testRegex: '(/__tests__/.*|(\\.|/)(test|spec))\\.tsx?$', globals: {
moduleFileExtensions: [ 'ts-jest': {
'ts', tsConfig: {
'tsx', target: 'es6',
'js',
'jsx'
],
globals: {
'ts-jest': {
tsConfig: {
"target": "es6",
// Needed because jest calls tsc in a way that doesn't // Needed because jest calls tsc in a way that doesn't
// inline const enums. // inline const enums.
"preserveConstEnums": true preserveConstEnums: true
} }
}
} }
} };
}

228
server/package-lock.json generated
View File

@ -456,9 +456,9 @@
"dev": true "dev": true
}, },
"@types/fs-extra": { "@types/fs-extra": {
"version": "8.0.1", "version": "8.1.0",
"resolved": "https://registry.npmjs.org/@types/fs-extra/-/fs-extra-8.0.1.tgz", "resolved": "https://registry.npmjs.org/@types/fs-extra/-/fs-extra-8.1.0.tgz",
"integrity": "sha512-J00cVDALmi/hJOYsunyT52Hva5TnJeKP5yd1r+mH/ZU0mbYZflR0Z5kw5kITtKTRYMhm1JMClOFYdHnQszEvqw==", "integrity": "sha512-UoOfVEzAUpeSPmjm7h1uk5MH6KZma2z2O7a75onTGjnNvAvMVrPzPL/vBbT65iIGHWj6rokwfmYcmxmlSf2uwg==",
"dev": true, "dev": true,
"requires": { "requires": {
"@types/node": "*" "@types/node": "*"
@ -490,24 +490,24 @@
} }
}, },
"@types/jest": { "@types/jest": {
"version": "24.0.24", "version": "24.9.1",
"resolved": "https://registry.npmjs.org/@types/jest/-/jest-24.0.24.tgz", "resolved": "https://registry.npmjs.org/@types/jest/-/jest-24.9.1.tgz",
"integrity": "sha512-vgaG968EDPSJPMunEDdZvZgvxYSmeH8wKqBlHSkBt1pV2XlLEVDzsj1ZhLuI4iG4Pv841tES61txSBF0obh4CQ==", "integrity": "sha512-Fb38HkXSVA4L8fGKEZ6le5bB8r6MRWlOCZbVuWZcmOMSCd2wCYOwN1ibj8daIoV9naq7aaOZjrLCoCMptKU/4Q==",
"dev": true, "dev": true,
"requires": { "requires": {
"jest-diff": "^24.3.0" "jest-diff": "^24.3.0"
} }
}, },
"@types/json-schema": { "@types/json-schema": {
"version": "7.0.3", "version": "7.0.4",
"resolved": "https://registry.npmjs.org/@types/json-schema/-/json-schema-7.0.3.tgz", "resolved": "https://registry.npmjs.org/@types/json-schema/-/json-schema-7.0.4.tgz",
"integrity": "sha512-Il2DtDVRGDcqjDtE+rF8iqg1CArehSK84HZJCT7AMITlyXRBpuPhqGLDQMowraqqu1coEaimg4ZOqggt6L6L+A==", "integrity": "sha512-8+KAKzEvSUdeo+kmqnKrqgeE+LcA0tjYWFY7RPProVYwnqDjukzO+3b6dLD56rYX5TdWejnEOLJYOIeh4CXKuA==",
"dev": true "dev": true
}, },
"@types/node": { "@types/node": {
"version": "12.12.21", "version": "12.12.28",
"resolved": "https://registry.npmjs.org/@types/node/-/node-12.12.21.tgz", "resolved": "https://registry.npmjs.org/@types/node/-/node-12.12.28.tgz",
"integrity": "sha512-8sRGhbpU+ck1n0PGAUgVrWrWdjSW2aqNeyC15W88GRsMpSwzv6RJGlLhE7s2RhVSOdyDmxbqlWSeThq4/7xqlA==", "integrity": "sha512-g73GJYJDXgf0jqg+P9S8h2acWbDXNkoCX8DLtJVu7Fkn788pzQ/oJsrdJz/2JejRf/SjfZaAhsw+3nd1D5EWGg==",
"dev": true "dev": true
}, },
"@types/stack-utils": { "@types/stack-utils": {
@ -532,12 +532,12 @@
"dev": true "dev": true
}, },
"@typescript-eslint/eslint-plugin": { "@typescript-eslint/eslint-plugin": {
"version": "2.12.0", "version": "2.21.0",
"resolved": "https://registry.npmjs.org/@typescript-eslint/eslint-plugin/-/eslint-plugin-2.12.0.tgz", "resolved": "https://registry.npmjs.org/@typescript-eslint/eslint-plugin/-/eslint-plugin-2.21.0.tgz",
"integrity": "sha512-1t4r9rpLuEwl3hgt90jY18wJHSyb0E3orVL3DaqwmpiSDHmHiSspVsvsFF78BJ/3NNG3qmeso836jpuBWYziAA==", "integrity": "sha512-b5jjjDMxzcjh/Sbjuo7WyhrQmVJg0WipTHQgXh5Xwx10uYm6nPWqN1WGOsaNq4HR3Zh4wUx4IRQdDkCHwyewyw==",
"dev": true, "dev": true,
"requires": { "requires": {
"@typescript-eslint/experimental-utils": "2.12.0", "@typescript-eslint/experimental-utils": "2.21.0",
"eslint-utils": "^1.4.3", "eslint-utils": "^1.4.3",
"functional-red-black-tree": "^1.0.1", "functional-red-black-tree": "^1.0.1",
"regexpp": "^3.0.0", "regexpp": "^3.0.0",
@ -545,51 +545,39 @@
} }
}, },
"@typescript-eslint/experimental-utils": { "@typescript-eslint/experimental-utils": {
"version": "2.12.0", "version": "2.21.0",
"resolved": "https://registry.npmjs.org/@typescript-eslint/experimental-utils/-/experimental-utils-2.12.0.tgz", "resolved": "https://registry.npmjs.org/@typescript-eslint/experimental-utils/-/experimental-utils-2.21.0.tgz",
"integrity": "sha512-jv4gYpw5N5BrWF3ntROvCuLe1IjRenLy5+U57J24NbPGwZFAjhnM45qpq0nDH1y/AZMb3Br25YiNVwyPbz6RkA==", "integrity": "sha512-olKw9JP/XUkav4lq0I7S1mhGgONJF9rHNhKFn9wJlpfRVjNo3PPjSvybxEldvCXnvD+WAshSzqH5cEjPp9CsBA==",
"dev": true, "dev": true,
"requires": { "requires": {
"@types/json-schema": "^7.0.3", "@types/json-schema": "^7.0.3",
"@typescript-eslint/typescript-estree": "2.12.0", "@typescript-eslint/typescript-estree": "2.21.0",
"eslint-scope": "^5.0.0" "eslint-scope": "^5.0.0"
},
"dependencies": {
"eslint-scope": {
"version": "5.0.0",
"resolved": "https://registry.npmjs.org/eslint-scope/-/eslint-scope-5.0.0.tgz",
"integrity": "sha512-oYrhJW7S0bxAFDvWqzvMPRm6pcgcnWc4QnofCAqRTRfQC0JcwenzGglTtsLyIuuWFfkqDG9vz67cnttSd53djw==",
"dev": true,
"requires": {
"esrecurse": "^4.1.0",
"estraverse": "^4.1.1"
}
}
} }
}, },
"@typescript-eslint/parser": { "@typescript-eslint/parser": {
"version": "2.12.0", "version": "2.21.0",
"resolved": "https://registry.npmjs.org/@typescript-eslint/parser/-/parser-2.12.0.tgz", "resolved": "https://registry.npmjs.org/@typescript-eslint/parser/-/parser-2.21.0.tgz",
"integrity": "sha512-lPdkwpdzxEfjI8TyTzZqPatkrswLSVu4bqUgnB03fHSOwpC7KSerPgJRgIAf11UGNf7HKjJV6oaPZI4AghLU6g==", "integrity": "sha512-VrmbdrrrvvI6cPPOG7uOgGUFXNYTiSbnRq8ZMyuGa4+qmXJXVLEEz78hKuqupvkpwJQNk1Ucz1TenrRP90gmBg==",
"dev": true, "dev": true,
"requires": { "requires": {
"@types/eslint-visitor-keys": "^1.0.0", "@types/eslint-visitor-keys": "^1.0.0",
"@typescript-eslint/experimental-utils": "2.12.0", "@typescript-eslint/experimental-utils": "2.21.0",
"@typescript-eslint/typescript-estree": "2.12.0", "@typescript-eslint/typescript-estree": "2.21.0",
"eslint-visitor-keys": "^1.1.0" "eslint-visitor-keys": "^1.1.0"
} }
}, },
"@typescript-eslint/typescript-estree": { "@typescript-eslint/typescript-estree": {
"version": "2.12.0", "version": "2.21.0",
"resolved": "https://registry.npmjs.org/@typescript-eslint/typescript-estree/-/typescript-estree-2.12.0.tgz", "resolved": "https://registry.npmjs.org/@typescript-eslint/typescript-estree/-/typescript-estree-2.21.0.tgz",
"integrity": "sha512-rGehVfjHEn8Frh9UW02ZZIfJs6SIIxIu/K1bbci8rFfDE/1lQ8krIJy5OXOV3DVnNdDPtoiPOdEANkLMrwXbiQ==", "integrity": "sha512-NC/nogZNb9IK2MEFQqyDBAciOT8Lp8O3KgAfvHx2Skx6WBo+KmDqlU3R9KxHONaijfTIKtojRe3SZQyMjr3wBw==",
"dev": true, "dev": true,
"requires": { "requires": {
"debug": "^4.1.1", "debug": "^4.1.1",
"eslint-visitor-keys": "^1.1.0", "eslint-visitor-keys": "^1.1.0",
"glob": "^7.1.6", "glob": "^7.1.6",
"is-glob": "^4.0.1", "is-glob": "^4.0.1",
"lodash.unescape": "4.0.1", "lodash": "^4.17.15",
"semver": "^6.3.0", "semver": "^6.3.0",
"tsutils": "^3.17.1" "tsutils": "^3.17.1"
}, },
@ -1531,9 +1519,9 @@
} }
}, },
"chownr": { "chownr": {
"version": "1.1.3", "version": "1.1.4",
"resolved": "https://registry.npmjs.org/chownr/-/chownr-1.1.3.tgz", "resolved": "https://registry.npmjs.org/chownr/-/chownr-1.1.4.tgz",
"integrity": "sha512-i70fVHhmV3DtTl6nqvZOnIjbY0Pe4kAUjwHj8z0zAdgBtYrJyYwLKCCuRBQ5ppkyL0AkN7HKRnETdmdp1zqNXw==", "integrity": "sha512-jJ0bqzaylmJtVnNgzTeSOs8DPavpbYgEr/b0YL8/2GO3xJEhInFmhKMUnEJQjZumK7KXGFhUy89PrsJWlakBVg==",
"dev": true "dev": true
}, },
"chrome-trace-event": { "chrome-trace-event": {
@ -1660,9 +1648,9 @@
} }
}, },
"commander": { "commander": {
"version": "2.20.1", "version": "2.20.3",
"resolved": "https://registry.npmjs.org/commander/-/commander-2.20.1.tgz", "resolved": "https://registry.npmjs.org/commander/-/commander-2.20.3.tgz",
"integrity": "sha512-cCuLsMhJeWQ/ZpsFTbE765kvVfoeSddc4nU3up4fV+fDBcfUXnbITJ+JzhkdjzOqhURjZgujxaioam4RM9yGUg==", "integrity": "sha512-GpVkmM8vF2vQUkj2LvZmD35JxeJOLCwJ9cUkugyk2nuhbv3+mJvpLYYt+0+USMxE+oj+ey/lJEnhZw75x/OMcQ==",
"dev": true "dev": true
}, },
"commondir": { "commondir": {
@ -2109,6 +2097,7 @@
"version": "1.14.2", "version": "1.14.2",
"resolved": "https://registry.npmjs.org/es-abstract/-/es-abstract-1.14.2.tgz", "resolved": "https://registry.npmjs.org/es-abstract/-/es-abstract-1.14.2.tgz",
"integrity": "sha512-DgoQmbpFNOofkjJtKwr87Ma5EW4Dc8fWhD0R+ndq7Oc456ivUfGOOP6oAZTTKl5/CcNMP+EN+e3/iUzgE0veZg==", "integrity": "sha512-DgoQmbpFNOofkjJtKwr87Ma5EW4Dc8fWhD0R+ndq7Oc456ivUfGOOP6oAZTTKl5/CcNMP+EN+e3/iUzgE0veZg==",
"dev": true,
"requires": { "requires": {
"es-to-primitive": "^1.2.0", "es-to-primitive": "^1.2.0",
"function-bind": "^1.1.1", "function-bind": "^1.1.1",
@ -2126,6 +2115,7 @@
"version": "1.2.0", "version": "1.2.0",
"resolved": "https://registry.npmjs.org/es-to-primitive/-/es-to-primitive-1.2.0.tgz", "resolved": "https://registry.npmjs.org/es-to-primitive/-/es-to-primitive-1.2.0.tgz",
"integrity": "sha512-qZryBOJjV//LaxLTV6UC//WewneB3LcXOL9NP++ozKVXsIIIpm/2c13UDiD9Jp2eThsecw9m3jPqDwTyobcdbg==", "integrity": "sha512-qZryBOJjV//LaxLTV6UC//WewneB3LcXOL9NP++ozKVXsIIIpm/2c13UDiD9Jp2eThsecw9m3jPqDwTyobcdbg==",
"dev": true,
"requires": { "requires": {
"is-callable": "^1.1.4", "is-callable": "^1.1.4",
"is-date-object": "^1.0.1", "is-date-object": "^1.0.1",
@ -2263,9 +2253,9 @@
} }
}, },
"eslint-scope": { "eslint-scope": {
"version": "4.0.3", "version": "5.0.0",
"resolved": "https://registry.npmjs.org/eslint-scope/-/eslint-scope-4.0.3.tgz", "resolved": "https://registry.npmjs.org/eslint-scope/-/eslint-scope-5.0.0.tgz",
"integrity": "sha512-p7VutNr1O/QrxysMo3E45FjYDTeXBy0iTltPFNSqKAIfjDSXC+4dj+qfyuD8bfAXrW/y6lW3O76VaYNPKfpKrg==", "integrity": "sha512-oYrhJW7S0bxAFDvWqzvMPRm6pcgcnWc4QnofCAqRTRfQC0JcwenzGglTtsLyIuuWFfkqDG9vz67cnttSd53djw==",
"dev": true, "dev": true,
"requires": { "requires": {
"esrecurse": "^4.1.0", "esrecurse": "^4.1.0",
@ -2343,9 +2333,9 @@
"dev": true "dev": true
}, },
"events": { "events": {
"version": "3.0.0", "version": "3.1.0",
"resolved": "https://registry.npmjs.org/events/-/events-3.0.0.tgz", "resolved": "https://registry.npmjs.org/events/-/events-3.1.0.tgz",
"integrity": "sha512-Dc381HFWJzEOhQ+d8pkNon++bk9h6cdAoAj4iE6Q4y6xgTzySWXlKn05/TVNpjnfRqi/X0EpJEJohPjNI3zpVA==", "integrity": "sha512-Rv+u8MLHNOdMjTAFeT3nCjHn2aGlx435FP/sDHNaRhDEMwyI/aB22Kj2qIN8R0cw3z28psEQLYwxVKLsKrMgWg==",
"dev": true "dev": true
}, },
"evp_bytestokey": { "evp_bytestokey": {
@ -3342,7 +3332,8 @@
"function-bind": { "function-bind": {
"version": "1.1.1", "version": "1.1.1",
"resolved": "https://registry.npmjs.org/function-bind/-/function-bind-1.1.1.tgz", "resolved": "https://registry.npmjs.org/function-bind/-/function-bind-1.1.1.tgz",
"integrity": "sha512-yIovAzMX49sF8Yl58fSCWJ5svSLuaibPxXQJFLmBObTuCr0Mf1KiPopGM9NiFjiYBCbfaa2Fh6breQ6ANVTI0A==" "integrity": "sha512-yIovAzMX49sF8Yl58fSCWJ5svSLuaibPxXQJFLmBObTuCr0Mf1KiPopGM9NiFjiYBCbfaa2Fh6breQ6ANVTI0A==",
"dev": true
}, },
"functional-red-black-tree": { "functional-red-black-tree": {
"version": "1.0.1", "version": "1.0.1",
@ -3487,6 +3478,7 @@
"version": "1.0.3", "version": "1.0.3",
"resolved": "https://registry.npmjs.org/has/-/has-1.0.3.tgz", "resolved": "https://registry.npmjs.org/has/-/has-1.0.3.tgz",
"integrity": "sha512-f2dvO0VU6Oej7RkWJGrehjbzMAjFp5/VKPp5tTpWIV4JHHZK1/BxbFRtf/siA2SWTe09caDmVtYYzWEIbBS4zw==", "integrity": "sha512-f2dvO0VU6Oej7RkWJGrehjbzMAjFp5/VKPp5tTpWIV4JHHZK1/BxbFRtf/siA2SWTe09caDmVtYYzWEIbBS4zw==",
"dev": true,
"requires": { "requires": {
"function-bind": "^1.1.1" "function-bind": "^1.1.1"
} }
@ -3499,7 +3491,8 @@
"has-symbols": { "has-symbols": {
"version": "1.0.0", "version": "1.0.0",
"resolved": "https://registry.npmjs.org/has-symbols/-/has-symbols-1.0.0.tgz", "resolved": "https://registry.npmjs.org/has-symbols/-/has-symbols-1.0.0.tgz",
"integrity": "sha1-uhqPGvKg/DllD1yFA2dwQSIGO0Q=" "integrity": "sha1-uhqPGvKg/DllD1yFA2dwQSIGO0Q=",
"dev": true
}, },
"has-value": { "has-value": {
"version": "1.0.0", "version": "1.0.0",
@ -3843,7 +3836,8 @@
"is-callable": { "is-callable": {
"version": "1.1.4", "version": "1.1.4",
"resolved": "https://registry.npmjs.org/is-callable/-/is-callable-1.1.4.tgz", "resolved": "https://registry.npmjs.org/is-callable/-/is-callable-1.1.4.tgz",
"integrity": "sha512-r5p9sxJjYnArLjObpjA4xu5EKI3CuKHkJXMhT7kwbpUyIFD1n5PMAsoPvWnvtZiNz7LjkYDRZhd7FlI0eMijEA==" "integrity": "sha512-r5p9sxJjYnArLjObpjA4xu5EKI3CuKHkJXMhT7kwbpUyIFD1n5PMAsoPvWnvtZiNz7LjkYDRZhd7FlI0eMijEA==",
"dev": true
}, },
"is-ci": { "is-ci": {
"version": "2.0.0", "version": "2.0.0",
@ -3877,7 +3871,8 @@
"is-date-object": { "is-date-object": {
"version": "1.0.1", "version": "1.0.1",
"resolved": "https://registry.npmjs.org/is-date-object/-/is-date-object-1.0.1.tgz", "resolved": "https://registry.npmjs.org/is-date-object/-/is-date-object-1.0.1.tgz",
"integrity": "sha1-mqIOtq7rv/d/vTPnTKAbM1gdOhY=" "integrity": "sha1-mqIOtq7rv/d/vTPnTKAbM1gdOhY=",
"dev": true
}, },
"is-descriptor": { "is-descriptor": {
"version": "0.1.6", "version": "0.1.6",
@ -3935,11 +3930,11 @@
} }
}, },
"is-nan": { "is-nan": {
"version": "1.2.1", "version": "1.3.0",
"resolved": "https://registry.npmjs.org/is-nan/-/is-nan-1.2.1.tgz", "resolved": "https://registry.npmjs.org/is-nan/-/is-nan-1.3.0.tgz",
"integrity": "sha1-n69ltvttskt/XAYoR16nH5iEAeI=", "integrity": "sha512-z7bbREymOqt2CCaZVly8aC4ML3Xhfi0ekuOnjO2L8vKdl+CttdVoGZQhd4adMFAsxQ5VeRVwORs4tU8RH+HFtQ==",
"requires": { "requires": {
"define-properties": "^1.1.1" "define-properties": "^1.1.3"
} }
}, },
"is-number": { "is-number": {
@ -3981,6 +3976,7 @@
"version": "1.0.4", "version": "1.0.4",
"resolved": "https://registry.npmjs.org/is-regex/-/is-regex-1.0.4.tgz", "resolved": "https://registry.npmjs.org/is-regex/-/is-regex-1.0.4.tgz",
"integrity": "sha1-VRdIm1RwkbCTDglWVM7SXul+lJE=", "integrity": "sha1-VRdIm1RwkbCTDglWVM7SXul+lJE=",
"dev": true,
"requires": { "requires": {
"has": "^1.0.1" "has": "^1.0.1"
} }
@ -3995,6 +3991,7 @@
"version": "1.0.2", "version": "1.0.2",
"resolved": "https://registry.npmjs.org/is-symbol/-/is-symbol-1.0.2.tgz", "resolved": "https://registry.npmjs.org/is-symbol/-/is-symbol-1.0.2.tgz",
"integrity": "sha512-HS8bZ9ox60yCJLH9snBpIwv9pYUAkcuLhSA1oero1UB5y9aiQpRA8y2ex945AOtCZL1lJDeIk3G5LthswI46Lw==", "integrity": "sha512-HS8bZ9ox60yCJLH9snBpIwv9pYUAkcuLhSA1oero1UB5y9aiQpRA8y2ex945AOtCZL1lJDeIk3G5LthswI46Lw==",
"dev": true,
"requires": { "requires": {
"has-symbols": "^1.0.0" "has-symbols": "^1.0.0"
} }
@ -4802,12 +4799,6 @@
"integrity": "sha1-7dFMgk4sycHgsKG0K7UhBRakJDg=", "integrity": "sha1-7dFMgk4sycHgsKG0K7UhBRakJDg=",
"dev": true "dev": true
}, },
"lodash.unescape": {
"version": "4.0.1",
"resolved": "https://registry.npmjs.org/lodash.unescape/-/lodash.unescape-4.0.1.tgz",
"integrity": "sha1-vyJJiGzlFM2hEvrpIYzcBlIR/Jw=",
"dev": true
},
"loose-envify": { "loose-envify": {
"version": "1.4.0", "version": "1.4.0",
"resolved": "https://registry.npmjs.org/loose-envify/-/loose-envify-1.4.0.tgz", "resolved": "https://registry.npmjs.org/loose-envify/-/loose-envify-1.4.0.tgz",
@ -4845,9 +4836,9 @@
} }
}, },
"make-error": { "make-error": {
"version": "1.3.5", "version": "1.3.6",
"resolved": "https://registry.npmjs.org/make-error/-/make-error-1.3.5.tgz", "resolved": "https://registry.npmjs.org/make-error/-/make-error-1.3.6.tgz",
"integrity": "sha512-c3sIjNUow0+8swNwVpqoH4YCShKNFkMaw6oH1mNS2haDZQqkeZFlHS3dhoeEbKKmJB4vXpJucU6oH75aDYeE9g==", "integrity": "sha512-s8UhlNe7vPKomQhC1qFelMokr/Sc3AgNbso3n74mVPA5LTZwkB9NlXf4XPamLxJE8h0gh73rM94xvwRT2CVInw==",
"dev": true "dev": true
}, },
"makeerror": { "makeerror": {
@ -5322,12 +5313,13 @@
"object-inspect": { "object-inspect": {
"version": "1.6.0", "version": "1.6.0",
"resolved": "https://registry.npmjs.org/object-inspect/-/object-inspect-1.6.0.tgz", "resolved": "https://registry.npmjs.org/object-inspect/-/object-inspect-1.6.0.tgz",
"integrity": "sha512-GJzfBZ6DgDAmnuaM3104jR4s1Myxr3Y3zfIyN4z3UdqN69oSRacNK8UhnobDdC+7J2AHCjGwxQubNJfE70SXXQ==" "integrity": "sha512-GJzfBZ6DgDAmnuaM3104jR4s1Myxr3Y3zfIyN4z3UdqN69oSRacNK8UhnobDdC+7J2AHCjGwxQubNJfE70SXXQ==",
"dev": true
}, },
"object-is": { "object-is": {
"version": "1.0.1", "version": "1.0.2",
"resolved": "https://registry.npmjs.org/object-is/-/object-is-1.0.1.tgz", "resolved": "https://registry.npmjs.org/object-is/-/object-is-1.0.2.tgz",
"integrity": "sha1-CqYOyZiaCz7Xlc9NBvYs8a1lObY=" "integrity": "sha512-Epah+btZd5wrrfjkJZq1AOB9O6OxUQto45hzFd7lXGrpHPGE0W1k+426yrZV+k6NJOzLNNW/nVsmZdIWsAqoOQ=="
}, },
"object-keys": { "object-keys": {
"version": "1.1.1", "version": "1.1.1",
@ -5343,17 +5335,6 @@
"isobject": "^3.0.0" "isobject": "^3.0.0"
} }
}, },
"object.entries": {
"version": "1.1.0",
"resolved": "https://registry.npmjs.org/object.entries/-/object.entries-1.1.0.tgz",
"integrity": "sha512-l+H6EQ8qzGRxbkHOd5I/aHRhHDKoQXQ8g0BYt4uSweQU1/J6dZUOyWh9a2Vky35YCKjzmgxOzta2hH6kf9HuXA==",
"requires": {
"define-properties": "^1.1.3",
"es-abstract": "^1.12.0",
"function-bind": "^1.1.1",
"has": "^1.0.3"
}
},
"object.getownpropertydescriptors": { "object.getownpropertydescriptors": {
"version": "2.0.3", "version": "2.0.3",
"resolved": "https://registry.npmjs.org/object.getownpropertydescriptors/-/object.getownpropertydescriptors-2.0.3.tgz", "resolved": "https://registry.npmjs.org/object.getownpropertydescriptors/-/object.getownpropertydescriptors-2.0.3.tgz",
@ -5512,9 +5493,9 @@
"dev": true "dev": true
}, },
"pako": { "pako": {
"version": "1.0.10", "version": "1.0.11",
"resolved": "https://registry.npmjs.org/pako/-/pako-1.0.10.tgz", "resolved": "https://registry.npmjs.org/pako/-/pako-1.0.11.tgz",
"integrity": "sha512-0DTvPVU3ed8+HNXOu5Bs+o//Mbdj9VNQMUOe9oKCwh8l0GNwpTDMKCWbRjgtD291AWnkAgkqA/LOnQS8AmS1tw==", "integrity": "sha512-4hLB8Py4zZce5s4yd9XzopqwVv/yGNhV1Bl8NTmCq1763HeK2+EwVTv+leGeL13Dnh2wfbqowVPXCIO0z4taYw==",
"dev": true "dev": true
}, },
"parallel-transform": { "parallel-transform": {
@ -6654,6 +6635,7 @@
"version": "2.1.0", "version": "2.1.0",
"resolved": "https://registry.npmjs.org/string.prototype.trimleft/-/string.prototype.trimleft-2.1.0.tgz", "resolved": "https://registry.npmjs.org/string.prototype.trimleft/-/string.prototype.trimleft-2.1.0.tgz",
"integrity": "sha512-FJ6b7EgdKxxbDxc79cOlok6Afd++TTs5szo+zJTUyow3ycrRfJVE2pq3vcN53XexvKZu/DJMDfeI/qMiZTrjTw==", "integrity": "sha512-FJ6b7EgdKxxbDxc79cOlok6Afd++TTs5szo+zJTUyow3ycrRfJVE2pq3vcN53XexvKZu/DJMDfeI/qMiZTrjTw==",
"dev": true,
"requires": { "requires": {
"define-properties": "^1.1.3", "define-properties": "^1.1.3",
"function-bind": "^1.1.1" "function-bind": "^1.1.1"
@ -6663,6 +6645,7 @@
"version": "2.1.0", "version": "2.1.0",
"resolved": "https://registry.npmjs.org/string.prototype.trimright/-/string.prototype.trimright-2.1.0.tgz", "resolved": "https://registry.npmjs.org/string.prototype.trimright/-/string.prototype.trimright-2.1.0.tgz",
"integrity": "sha512-fXZTSV55dNBwv16uw+hh5jkghxSnc5oHq+5K/gXgizHwAvMetdAJlHqqoFC1FSDVPYWLkAKl2cxpUT41sV7nSg==", "integrity": "sha512-fXZTSV55dNBwv16uw+hh5jkghxSnc5oHq+5K/gXgizHwAvMetdAJlHqqoFC1FSDVPYWLkAKl2cxpUT41sV7nSg==",
"dev": true,
"requires": { "requires": {
"define-properties": "^1.1.3", "define-properties": "^1.1.3",
"function-bind": "^1.1.1" "function-bind": "^1.1.1"
@ -6737,9 +6720,9 @@
"dev": true "dev": true
}, },
"terser": { "terser": {
"version": "4.4.3", "version": "4.6.3",
"resolved": "https://registry.npmjs.org/terser/-/terser-4.4.3.tgz", "resolved": "https://registry.npmjs.org/terser/-/terser-4.6.3.tgz",
"integrity": "sha512-0ikKraVtRDKGzHrzkCv5rUNDzqlhmhowOBqC0XqUHFpW+vJ45+20/IFBcebwKfiS2Z9fJin6Eo+F1zLZsxi8RA==", "integrity": "sha512-Lw+ieAXmY69d09IIc/yqeBqXpEQIpDGZqT34ui1QWXIUpR2RjbqEkT8X7Lgex19hslSqcWM5iMN2kM11eMsESQ==",
"dev": true, "dev": true,
"requires": { "requires": {
"commander": "^2.20.0", "commander": "^2.20.0",
@ -6902,9 +6885,9 @@
} }
}, },
"ts-jest": { "ts-jest": {
"version": "24.2.0", "version": "24.3.0",
"resolved": "https://registry.npmjs.org/ts-jest/-/ts-jest-24.2.0.tgz", "resolved": "https://registry.npmjs.org/ts-jest/-/ts-jest-24.3.0.tgz",
"integrity": "sha512-Yc+HLyldlIC9iIK8xEN7tV960Or56N49MDP7hubCZUeI7EbIOTsas6rXCMB4kQjLACJ7eDOF4xWEO5qumpKsag==", "integrity": "sha512-Hb94C/+QRIgjVZlJyiWwouYUF+siNJHJHknyspaOcZ+OQAIdFG/UrdQVXw/0B8Z3No34xkUXZJpOTy9alOWdVQ==",
"dev": true, "dev": true,
"requires": { "requires": {
"bs-logger": "0.x", "bs-logger": "0.x",
@ -7058,9 +7041,9 @@
"dev": true "dev": true
}, },
"typescript": { "typescript": {
"version": "3.7.3", "version": "3.8.2",
"resolved": "https://registry.npmjs.org/typescript/-/typescript-3.7.3.tgz", "resolved": "https://registry.npmjs.org/typescript/-/typescript-3.8.2.tgz",
"integrity": "sha512-Mcr/Qk7hXqFBXMN7p7Lusj1ktCBydylfQM/FZCk5glCNQJrCUKPkMHdo9R0MTFWsC/4kPFvDS0fDPvukfCkFsw==", "integrity": "sha512-EgOVgL/4xfVrCMbhYKUQTdF37SQn4Iw73H5BgCrF1Abdun7Kwy/QZsE/ssAy0y4LxBbvua3PIbFsbRczWWnDdQ==",
"dev": true "dev": true
}, },
"typescript-char": { "typescript-char": {
@ -7215,14 +7198,13 @@
"dev": true "dev": true
}, },
"util": { "util": {
"version": "0.12.1", "version": "0.12.2",
"resolved": "https://registry.npmjs.org/util/-/util-0.12.1.tgz", "resolved": "https://registry.npmjs.org/util/-/util-0.12.2.tgz",
"integrity": "sha512-MREAtYOp+GTt9/+kwf00IYoHZyjM8VU4aVrkzUlejyqaIjd2GztVl5V9hGXKlvBKE3gENn/FMfHE5v6hElXGcQ==", "integrity": "sha512-XE+MkWQvglYa+IOfBt5UFG93EmncEMP23UqpgDvVZVFBPxwmkK10QRp6pgU4xICPnWRf/t0zPv4noYSUq9gqUQ==",
"requires": { "requires": {
"inherits": "^2.0.3", "inherits": "^2.0.3",
"is-arguments": "^1.0.4", "is-arguments": "^1.0.4",
"is-generator-function": "^1.0.7", "is-generator-function": "^1.0.7",
"object.entries": "^1.1.0",
"safe-buffer": "^5.1.2" "safe-buffer": "^5.1.2"
}, },
"dependencies": { "dependencies": {
@ -7289,9 +7271,9 @@
"dev": true "dev": true
}, },
"vscode-jsonrpc": { "vscode-jsonrpc": {
"version": "5.0.0-next.2", "version": "5.0.1",
"resolved": "https://registry.npmjs.org/vscode-jsonrpc/-/vscode-jsonrpc-5.0.0-next.2.tgz", "resolved": "https://registry.npmjs.org/vscode-jsonrpc/-/vscode-jsonrpc-5.0.1.tgz",
"integrity": "sha512-Q3/jabZUNviCG9hhF6hHWjhrABevPF9mv0aiE2j8BYCAP2k+aHTpjMyk+04MzaAqWYwXdQuZkLSbcYCCqbzJLg==" "integrity": "sha512-JvONPptw3GAQGXlVV2utDcHx0BiY34FupW/kI6mZ5x06ER5DdPG/tXWMVHjTNULF5uKPOUUD0SaXg5QaubJL0A=="
}, },
"vscode-languageserver": { "vscode-languageserver": {
"version": "5.3.0-next.10", "version": "5.3.0-next.10",
@ -7303,18 +7285,18 @@
} }
}, },
"vscode-languageserver-protocol": { "vscode-languageserver-protocol": {
"version": "3.15.0-next.9", "version": "3.15.3",
"resolved": "https://registry.npmjs.org/vscode-languageserver-protocol/-/vscode-languageserver-protocol-3.15.0-next.9.tgz", "resolved": "https://registry.npmjs.org/vscode-languageserver-protocol/-/vscode-languageserver-protocol-3.15.3.tgz",
"integrity": "sha512-b9PAxouMmtsLEe8ZjbIMPb7wRWPhckGfgjwZLmp/dWnaAuRPYtY3lGO0/rNbLc3jKIqCVlnEyYVFKalzDAzj0g==", "integrity": "sha512-zrMuwHOAQRhjDSnflWdJG+O2ztMWss8GqUUB8dXLR/FPenwkiBNkMIJJYfSN6sgskvsF0rHAoBowNQfbyZnnvw==",
"requires": { "requires": {
"vscode-jsonrpc": "^5.0.0-next.2", "vscode-jsonrpc": "^5.0.1",
"vscode-languageserver-types": "^3.15.0-next.5" "vscode-languageserver-types": "3.15.1"
} }
}, },
"vscode-languageserver-types": { "vscode-languageserver-types": {
"version": "3.15.0-next.5", "version": "3.15.1",
"resolved": "https://registry.npmjs.org/vscode-languageserver-types/-/vscode-languageserver-types-3.15.0-next.5.tgz", "resolved": "https://registry.npmjs.org/vscode-languageserver-types/-/vscode-languageserver-types-3.15.1.tgz",
"integrity": "sha512-7hrELhTeWieUgex3+6692KjCkcmO/+V/bFItM5MHGcBotzwmjEuXjapLLYTYhIspuJ1ibRSik5MhX5YwLpsPiw==" "integrity": "sha512-+a9MPUQrNGRrGU630OGbYVQ+11iOIovjCkqxajPa9w57Sd5ruK8WQNsslzpa0x/QJqC8kRc2DUxWjIFwoNm4ZQ=="
}, },
"vscode-textbuffer": { "vscode-textbuffer": {
"version": "1.0.0", "version": "1.0.0",
@ -7443,9 +7425,9 @@
"dev": true "dev": true
}, },
"webpack": { "webpack": {
"version": "4.41.3", "version": "4.41.6",
"resolved": "https://registry.npmjs.org/webpack/-/webpack-4.41.3.tgz", "resolved": "https://registry.npmjs.org/webpack/-/webpack-4.41.6.tgz",
"integrity": "sha512-EcNzP9jGoxpQAXq1VOoTet0ik7/VVU1MovIfcUSAjLowc7GhcQku/sOXALvq5nPpSei2HF6VRhibeJSC3i/Law==", "integrity": "sha512-yxXfV0Zv9WMGRD+QexkZzmGIh54bsvEs+9aRWxnN8erLWEOehAKUTeNBoUbA6HPEZPlRo7KDi2ZcNveoZgK9MA==",
"dev": true, "dev": true,
"requires": { "requires": {
"@webassemblyjs/ast": "1.8.5", "@webassemblyjs/ast": "1.8.5",
@ -7478,13 +7460,23 @@
"resolved": "https://registry.npmjs.org/acorn/-/acorn-6.4.0.tgz", "resolved": "https://registry.npmjs.org/acorn/-/acorn-6.4.0.tgz",
"integrity": "sha512-gac8OEcQ2Li1dxIEWGZzsp2BitJxwkwcOm0zHAJLcPJaVvm58FRnk6RkuLRpU1EujipU2ZFODv2P9DLMfnV8mw==", "integrity": "sha512-gac8OEcQ2Li1dxIEWGZzsp2BitJxwkwcOm0zHAJLcPJaVvm58FRnk6RkuLRpU1EujipU2ZFODv2P9DLMfnV8mw==",
"dev": true "dev": true
},
"eslint-scope": {
"version": "4.0.3",
"resolved": "https://registry.npmjs.org/eslint-scope/-/eslint-scope-4.0.3.tgz",
"integrity": "sha512-p7VutNr1O/QrxysMo3E45FjYDTeXBy0iTltPFNSqKAIfjDSXC+4dj+qfyuD8bfAXrW/y6lW3O76VaYNPKfpKrg==",
"dev": true,
"requires": {
"esrecurse": "^4.1.0",
"estraverse": "^4.1.1"
}
} }
} }
}, },
"webpack-cli": { "webpack-cli": {
"version": "3.3.10", "version": "3.3.11",
"resolved": "https://registry.npmjs.org/webpack-cli/-/webpack-cli-3.3.10.tgz", "resolved": "https://registry.npmjs.org/webpack-cli/-/webpack-cli-3.3.11.tgz",
"integrity": "sha512-u1dgND9+MXaEt74sJR4PR7qkPxXUSQ0RXYq8x1L6Jg1MYVEmGPrH6Ah6C4arD4r0J1P5HKjRqpab36k0eIzPqg==", "integrity": "sha512-dXlfuml7xvAFwYUPsrtQAA9e4DOe58gnzSxhgrO/ZM/gyXTBowrsYeubyN4mqGhYdpXMFNyQ6emjJS9M7OBd4g==",
"dev": true, "dev": true,
"requires": { "requires": {
"chalk": "2.4.2", "chalk": "2.4.2",

18
server/package.json
View File

@ -31,20 +31,20 @@
"@types/chalk": "^2.2.0", "@types/chalk": "^2.2.0",
"@types/chokidar": "^2.1.3", "@types/chokidar": "^2.1.3",
"@types/command-line-args": "^5.0.0", "@types/command-line-args": "^5.0.0",
"@types/fs-extra": "^8.0.1", "@types/fs-extra": "^8.1.0",
"@types/jest": "^24.0.24", "@types/jest": "^24.9.1",
"@types/node": "^12.12.21", "@types/node": "^12.12.28",
"@typescript-eslint/eslint-plugin": "^2.12.0", "@typescript-eslint/eslint-plugin": "^2.21.0",
"@typescript-eslint/parser": "^2.12.0", "@typescript-eslint/parser": "^2.21.0",
"eslint": "^6.8.0", "eslint": "^6.8.0",
"fs-extra": "^8.1.0", "fs-extra": "^8.1.0",
"jest": "^24.9.0", "jest": "^24.9.0",
"node-loader": "^0.6.0", "node-loader": "^0.6.0",
"ts-jest": "^24.2.0", "ts-jest": "^24.3.0",
"ts-loader": "^6.2.1", "ts-loader": "^6.2.1",
"typescript": "^3.7.3", "typescript": "^3.8.2",
"webpack": "^4.41.3", "webpack": "^4.41.6",
"webpack-cli": "^3.3.10" "webpack-cli": "^3.3.11"
}, },
"types": "out/main.d.ts", "types": "out/main.d.ts",
"main": "out/main.js" "main": "out/main.js"

16
server/src/analyzer/analyzerFileInfo.ts
View File

@ -1,12 +1,12 @@
/* /*
* analyzerFileInfo.ts * analyzerFileInfo.ts
* Copyright (c) Microsoft Corporation. * Copyright (c) Microsoft Corporation.
* Licensed under the MIT license. * Licensed under the MIT license.
* Author: Eric Traut * Author: Eric Traut
* *
* Information associated with a source file that is used * Information associated with a source file that is used
* by the binder and checker. * by the binder and checker.
*/ */
import { DiagnosticSettings, ExecutionEnvironment } from '../common/configOptions'; import { DiagnosticSettings, ExecutionEnvironment } from '../common/configOptions';
import { TextRangeDiagnosticSink } from '../common/diagnosticSink'; import { TextRangeDiagnosticSink } from '../common/diagnosticSink';

34
server/src/analyzer/analyzerNodeInfo.ts
View File

@ -1,17 +1,24 @@
/* /*
* analyzerNodeInfo.ts * analyzerNodeInfo.ts
* Copyright (c) Microsoft Corporation. * Copyright (c) Microsoft Corporation.
* Licensed under the MIT license. * Licensed under the MIT license.
* Author: Eric Traut * Author: Eric Traut
* *
* Defines objects that hang off the parse nodes in the parse tree. * Defines objects that hang off the parse nodes in the parse tree.
* It contains information collected during the binder phase that * It contains information collected during the binder phase that
* can be used for later analysis steps or for language services * can be used for later analysis steps or for language services
* (e.g. hover information). * (e.g. hover information).
*/ */
import { ClassNode, ExecutionScopeNode, FunctionNode, LambdaNode, ListComprehensionNode, import {
ModuleNode, ParseNode } from '../parser/parseNodes'; ClassNode,
ExecutionScopeNode,
FunctionNode,
LambdaNode,
ListComprehensionNode,
ModuleNode,
ParseNode
} from '../parser/parseNodes';
import { AnalyzerFileInfo } from './analyzerFileInfo'; import { AnalyzerFileInfo } from './analyzerFileInfo';
import { FlowFlags, FlowNode } from './codeFlow'; import { FlowFlags, FlowNode } from './codeFlow';
import { Declaration } from './declaration'; import { Declaration } from './declaration';
@ -50,8 +57,7 @@ interface AnalyzerNodeInfo {
codeFlowExpressions?: Map<string, string>; codeFlowExpressions?: Map<string, string>;
} }
export type ScopedNode = ModuleNode | ClassNode | FunctionNode | export type ScopedNode = ModuleNode | ClassNode | FunctionNode | LambdaNode | ListComprehensionNode;
LambdaNode | ListComprehensionNode;
// Cleans out all fields that are added by the analyzer phases // Cleans out all fields that are added by the analyzer phases
// (after the post-parse walker). // (after the post-parse walker).

519
server/src/analyzer/binder.ts
View File

@ -1,20 +1,20 @@
/* /*
* binder.ts * binder.ts
* Copyright (c) Microsoft Corporation. * Copyright (c) Microsoft Corporation.
* Licensed under the MIT license. * Licensed under the MIT license.
* Author: Eric Traut * Author: Eric Traut
* *
* A parse tree walker that performs basic name binding (creation of * A parse tree walker that performs basic name binding (creation of
* scopes and associated symbol tables). * scopes and associated symbol tables).
* The binder walks the parse tree by scopes starting at the module * The binder walks the parse tree by scopes starting at the module
* level. When a new scope is detected, it is pushed onto a list and * level. When a new scope is detected, it is pushed onto a list and
* walked separately at a later time. (The exception is a class scope, * walked separately at a later time. (The exception is a class scope,
* which is immediately walked.) Walking the tree in this manner * which is immediately walked.) Walking the tree in this manner
* simulates the order in which execution normally occurs in a Python * simulates the order in which execution normally occurs in a Python
* file. The binder attempts to statically detect runtime errors that * file. The binder attempts to statically detect runtime errors that
* would be reported by the python interpreter when executing the code. * would be reported by the python interpreter when executing the code.
* This binder doesn't perform any static type checking. * This binder doesn't perform any static type checking.
*/ */
import { Commands } from '../commands/commands'; import { Commands } from '../commands/commands';
import { DiagnosticLevel } from '../common/configOptions'; import { DiagnosticLevel } from '../common/configOptions';
@ -25,23 +25,79 @@ import { convertOffsetsToRange } from '../common/positionUtils';
import { PythonVersion } from '../common/pythonVersion'; import { PythonVersion } from '../common/pythonVersion';
import { getEmptyRange } from '../common/textRange'; import { getEmptyRange } from '../common/textRange';
import { TextRange } from '../common/textRange'; import { TextRange } from '../common/textRange';
import { ArgumentCategory, AssertNode, AssignmentExpressionNode, AssignmentNode, import {
AugmentedAssignmentNode, AwaitNode, BinaryOperationNode, BreakNode, ArgumentCategory,
CallNode, ClassNode, ContinueNode, DelNode, ExceptNode, ExpressionNode, ForNode, AssertNode,
FunctionNode, GlobalNode, IfNode, ImportAsNode, ImportFromNode, LambdaNode, AssignmentExpressionNode,
ListComprehensionNode, MemberAccessNode, ModuleNameNode, ModuleNode, NameNode, NonlocalNode, AssignmentNode,
ParseNode, ParseNodeType, RaiseNode, ReturnNode, StatementNode, StringListNode, AugmentedAssignmentNode,
SuiteNode, TernaryNode, TryNode, TypeAnnotationNode, AwaitNode,
UnaryOperationNode, WhileNode, WithNode, YieldFromNode, YieldNode } from '../parser/parseNodes'; BinaryOperationNode,
BreakNode,
CallNode,
ClassNode,
ContinueNode,
DelNode,
ExceptNode,
ExpressionNode,
ForNode,
FunctionNode,
GlobalNode,
IfNode,
ImportAsNode,
ImportFromNode,
LambdaNode,
ListComprehensionNode,
MemberAccessNode,
ModuleNameNode,
ModuleNode,
NameNode,
NonlocalNode,
ParseNode,
ParseNodeType,
RaiseNode,
ReturnNode,
StatementNode,
StringListNode,
SuiteNode,
TernaryNode,
TryNode,
TypeAnnotationNode,
UnaryOperationNode,
WhileNode,
WithNode,
YieldFromNode,
YieldNode
} from '../parser/parseNodes';
import * as StringTokenUtils from '../parser/stringTokenUtils'; import * as StringTokenUtils from '../parser/stringTokenUtils';
import { KeywordType, OperatorType } from '../parser/tokenizerTypes'; import { KeywordType, OperatorType } from '../parser/tokenizerTypes';
import { AnalyzerFileInfo, ImportLookupResult } from './analyzerFileInfo'; import { AnalyzerFileInfo, ImportLookupResult } from './analyzerFileInfo';
import * as AnalyzerNodeInfo from './analyzerNodeInfo'; import * as AnalyzerNodeInfo from './analyzerNodeInfo';
import { createKeyForReference, FlowAssignment, FlowAssignmentAlias, FlowCall, FlowCondition, import {
FlowFlags, FlowLabel, FlowNode, FlowPostFinally, FlowPreFinallyGate, FlowWildcardImport, createKeyForReference,
getUniqueFlowNodeId, isCodeFlowSupportedForReference } from './codeFlow'; FlowAssignment,
import { AliasDeclaration, ClassDeclaration, DeclarationType, FunctionDeclaration, FlowAssignmentAlias,
IntrinsicType, ModuleLoaderActions, ParameterDeclaration, VariableDeclaration } from './declaration'; FlowCall,
FlowCondition,
FlowFlags,
FlowLabel,
FlowNode,
FlowPostFinally,
FlowPreFinallyGate,
FlowWildcardImport,
getUniqueFlowNodeId,
isCodeFlowSupportedForReference
} from './codeFlow';
import {
AliasDeclaration,
ClassDeclaration,
DeclarationType,
FunctionDeclaration,
IntrinsicType,
ModuleLoaderActions,
ParameterDeclaration,
VariableDeclaration
} from './declaration';
import { ImplicitImport, ImportResult, ImportType } from './importResult'; import { ImplicitImport, ImportResult, ImportType } from './importResult';
import * as ParseTreeUtils from './parseTreeUtils'; import * as ParseTreeUtils from './parseTreeUtils';
import { ParseTreeWalker } from './parseTreeWalker'; import { ParseTreeWalker } from './parseTreeWalker';
@ -147,29 +203,31 @@ export class Binder extends ParseTreeWalker {
// binding the builtins module itself. // binding the builtins module itself.
const isBuiltInModule = this._fileInfo.builtinsScope === undefined; const isBuiltInModule = this._fileInfo.builtinsScope === undefined;
this._createNewScope(isBuiltInModule ? ScopeType.Builtin : ScopeType.Module, this._createNewScope(
this._fileInfo.builtinsScope, () => { isBuiltInModule ? ScopeType.Builtin : ScopeType.Module,
this._fileInfo.builtinsScope,
() => {
AnalyzerNodeInfo.setScope(node, this._currentScope);
AnalyzerNodeInfo.setScope(node, this._currentScope); // Bind implicit names.
// List taken from https://docs.python.org/3/reference/import.html#__name__
this._addBuiltInSymbolToCurrentScope('__doc__', node, 'str');
this._addBuiltInSymbolToCurrentScope('__name__', node, 'str');
this._addBuiltInSymbolToCurrentScope('__loader__', node, 'Any');
this._addBuiltInSymbolToCurrentScope('__package__', node, 'str');
this._addBuiltInSymbolToCurrentScope('__spec__', node, 'Any');
this._addBuiltInSymbolToCurrentScope('__path__', node, 'Iterable[str]');
this._addBuiltInSymbolToCurrentScope('__file__', node, 'str');
this._addBuiltInSymbolToCurrentScope('__cached__', node, 'str');
// Bind implicit names. // Create a start node for the module.
// List taken from https://docs.python.org/3/reference/import.html#__name__ this._currentFlowNode = this._createStartFlowNode();
this._addBuiltInSymbolToCurrentScope('__doc__', node, 'str');
this._addBuiltInSymbolToCurrentScope('__name__', node, 'str');
this._addBuiltInSymbolToCurrentScope('__loader__', node, 'Any');
this._addBuiltInSymbolToCurrentScope('__package__', node, 'str');
this._addBuiltInSymbolToCurrentScope('__spec__', node, 'Any');
this._addBuiltInSymbolToCurrentScope('__path__', node, 'Iterable[str]');
this._addBuiltInSymbolToCurrentScope('__file__', node, 'str');
this._addBuiltInSymbolToCurrentScope('__cached__', node, 'str');
// Create a start node for the module. this._walkStatementsAndReportUnreachable(node.statements);
this._currentFlowNode = this._createStartFlowNode();
this._walkStatementsAndReportUnreachable(node.statements); AnalyzerNodeInfo.setCodeFlowExpressions(node, this._currentExecutionScopeReferenceMap);
}
AnalyzerNodeInfo.setCodeFlowExpressions(node, this._currentExecutionScopeReferenceMap); );
});
// Perform all analysis that was deferred during the first pass. // Perform all analysis that was deferred during the first pass.
this._bindDeferred(); this._bindDeferred();
@ -197,15 +255,20 @@ export class Binder extends ParseTreeWalker {
if (importResult) { if (importResult) {
if (!importResult.isImportFound) { if (!importResult.isImportFound) {
this._addDiagnostic(this._fileInfo.diagnosticSettings.reportMissingImports, this._addDiagnostic(
this._fileInfo.diagnosticSettings.reportMissingImports,
DiagnosticRule.reportMissingImports, DiagnosticRule.reportMissingImports,
`Import '${ importResult.importName }' could not be resolved`, node); `Import '${importResult.importName}' could not be resolved`,
node
);
} else if (importResult.importType === ImportType.ThirdParty) { } else if (importResult.importType === ImportType.ThirdParty) {
if (!importResult.isStubFile) { if (!importResult.isStubFile) {
const diagnostic = this._addDiagnostic( const diagnostic = this._addDiagnostic(
this._fileInfo.diagnosticSettings.reportMissingTypeStubs, this._fileInfo.diagnosticSettings.reportMissingTypeStubs,
DiagnosticRule.reportMissingTypeStubs, DiagnosticRule.reportMissingTypeStubs,
`Stub file not found for '${ importResult.importName }'`, node); `Stub file not found for '${importResult.importName}'`,
node
);
if (diagnostic) { if (diagnostic) {
// Add a diagnostic action for resolving this diagnostic. // Add a diagnostic action for resolving this diagnostic.
const createTypeStubAction: CreateTypeStubFileAction = { const createTypeStubAction: CreateTypeStubFileAction = {
@ -228,8 +291,7 @@ export class Binder extends ParseTreeWalker {
type: DeclarationType.Class, type: DeclarationType.Class,
node, node,
path: this._fileInfo.filePath, path: this._fileInfo.filePath,
range: convertOffsetsToRange(node.name.start, range: convertOffsetsToRange(node.name.start, TextRange.getEnd(node.name), this._fileInfo.lines)
TextRange.getEnd(node.name), this._fileInfo.lines)
}; };
const symbol = this._bindNameToScope(this._currentScope, node.name.value); const symbol = this._bindNameToScope(this._currentScope, node.name.value);
@ -237,7 +299,7 @@ export class Binder extends ParseTreeWalker {
symbol.addDeclaration(classDeclaration); symbol.addDeclaration(classDeclaration);
} }
// Stash the declaration in the parse node for later access. // Stash the declaration in the parse node for later access.
AnalyzerNodeInfo.setDeclaration(node, classDeclaration); AnalyzerNodeInfo.setDeclaration(node, classDeclaration);
this.walkMultiple(node.arguments); this.walkMultiple(node.arguments);
@ -277,8 +339,7 @@ export class Binder extends ParseTreeWalker {
isMethod: !!containingClassNode, isMethod: !!containingClassNode,
isGenerator: false, isGenerator: false,
path: this._fileInfo.filePath, path: this._fileInfo.filePath,
range: convertOffsetsToRange(node.name.start, TextRange.getEnd(node.name), range: convertOffsetsToRange(node.name.start, TextRange.getEnd(node.name), this._fileInfo.lines)
this._fileInfo.lines)
}; };
if (symbol) { if (symbol) {
@ -308,8 +369,10 @@ export class Binder extends ParseTreeWalker {
// the scope of the containing function or module when they execute. // the scope of the containing function or module when they execute.
let functionOrModuleNode: ParseNode | undefined = node.parent; let functionOrModuleNode: ParseNode | undefined = node.parent;
while (functionOrModuleNode) { while (functionOrModuleNode) {
if (functionOrModuleNode.nodeType === ParseNodeType.Module || if (
functionOrModuleNode.nodeType === ParseNodeType.Function) { functionOrModuleNode.nodeType === ParseNodeType.Module ||
functionOrModuleNode.nodeType === ParseNodeType.Function
) {
break; break;
} }
@ -358,8 +421,11 @@ export class Binder extends ParseTreeWalker {
type: DeclarationType.Parameter, type: DeclarationType.Parameter,
node: paramNode, node: paramNode,
path: this._fileInfo.filePath, path: this._fileInfo.filePath,
range: convertOffsetsToRange(paramNode.start, TextRange.getEnd(paramNode), range: convertOffsetsToRange(
this._fileInfo.lines) paramNode.start,
TextRange.getEnd(paramNode),
this._fileInfo.lines
)
}; };
symbol.addDeclaration(paramDeclaration); symbol.addDeclaration(paramDeclaration);
@ -421,8 +487,11 @@ export class Binder extends ParseTreeWalker {
type: DeclarationType.Parameter, type: DeclarationType.Parameter,
node: paramNode, node: paramNode,
path: this._fileInfo.filePath, path: this._fileInfo.filePath,
range: convertOffsetsToRange(paramNode.start, TextRange.getEnd(paramNode), range: convertOffsetsToRange(
this._fileInfo.lines) paramNode.start,
TextRange.getEnd(paramNode),
this._fileInfo.lines
)
}; };
symbol.addDeclaration(paramDeclaration); symbol.addDeclaration(paramDeclaration);
@ -478,9 +547,7 @@ export class Binder extends ParseTreeWalker {
const evaluationNode = ParseTreeUtils.getEvaluationNodeForAssignmentExpression(node); const evaluationNode = ParseTreeUtils.getEvaluationNodeForAssignmentExpression(node);
if (!evaluationNode) { if (!evaluationNode) {
this._addError( this._addError('Assignment expression must be within module, function or lambda', node);
'Assignment expression must be within module, function or lambda',
node);
} else { } else {
// Bind the name to the containing scope. This special logic is required // Bind the name to the containing scope. This special logic is required
// because of the behavior defined in PEP 572. Targets of assignment // because of the behavior defined in PEP 572. Targets of assignment
@ -496,9 +563,10 @@ export class Binder extends ParseTreeWalker {
const localSymbol = curScope.lookUpSymbol(node.name.value); const localSymbol = curScope.lookUpSymbol(node.name.value);
if (localSymbol) { if (localSymbol) {
this._addError( this._addError(
`Assignment expression target '${ node.name.value }' ` + `Assignment expression target '${node.name.value}' ` +
`cannot use same name as comprehension for target`, `cannot use same name as comprehension for target`,
node.name); node.name
);
break; break;
} }
@ -649,21 +717,23 @@ export class Binder extends ParseTreeWalker {
// Determine if the test condition is always true or always false. If so, // Determine if the test condition is always true or always false. If so,
// we can treat either the then or the else clause as unconditional. // we can treat either the then or the else clause as unconditional.
const constExprValue = StaticExpressions.evaluateStaticBoolLikeExpression( const constExprValue = StaticExpressions.evaluateStaticBoolLikeExpression(
node.testExpression, this._fileInfo.executionEnvironment); node.testExpression,
this._fileInfo.executionEnvironment
);
this._bindConditional(node.testExpression, thenLabel, elseLabel); this._bindConditional(node.testExpression, thenLabel, elseLabel);
// Handle the if clause. // Handle the if clause.
this._currentFlowNode = constExprValue === false ? this._currentFlowNode =
Binder._unreachableFlowNode : this._finishFlowLabel(thenLabel); constExprValue === false ? Binder._unreachableFlowNode : this._finishFlowLabel(thenLabel);
this.walk(node.ifSuite); this.walk(node.ifSuite);
this._addAntecedent(postIfLabel, this._currentFlowNode); this._addAntecedent(postIfLabel, this._currentFlowNode);
// Now handle the else clause if it's present. If there // Now handle the else clause if it's present. If there
// are chained "else if" statements, they'll be handled // are chained "else if" statements, they'll be handled
// recursively here. // recursively here.
this._currentFlowNode = constExprValue === true ? this._currentFlowNode =
Binder._unreachableFlowNode : this._finishFlowLabel(elseLabel); constExprValue === true ? Binder._unreachableFlowNode : this._finishFlowLabel(elseLabel);
if (node.elseSuite) { if (node.elseSuite) {
this.walk(node.elseSuite); this.walk(node.elseSuite);
} }
@ -681,7 +751,9 @@ export class Binder extends ParseTreeWalker {
// Determine if the test condition is always true or always false. If so, // Determine if the test condition is always true or always false. If so,
// we can treat either the while or the else clause as unconditional. // we can treat either the while or the else clause as unconditional.
const constExprValue = StaticExpressions.evaluateStaticBoolLikeExpression( const constExprValue = StaticExpressions.evaluateStaticBoolLikeExpression(
node.testExpression, this._fileInfo.executionEnvironment); node.testExpression,
this._fileInfo.executionEnvironment
);
const preLoopLabel = this._createLoopLabel(); const preLoopLabel = this._createLoopLabel();
this._addAntecedent(preLoopLabel, this._currentFlowNode); this._addAntecedent(preLoopLabel, this._currentFlowNode);
@ -690,15 +762,15 @@ export class Binder extends ParseTreeWalker {
this._bindConditional(node.testExpression, thenLabel, elseLabel); this._bindConditional(node.testExpression, thenLabel, elseLabel);
// Handle the while clause. // Handle the while clause.
this._currentFlowNode = constExprValue === false ? this._currentFlowNode =
Binder._unreachableFlowNode : this._finishFlowLabel(thenLabel); constExprValue === false ? Binder._unreachableFlowNode : this._finishFlowLabel(thenLabel);
this._bindLoopStatement(preLoopLabel, postWhileLabel, () => { this._bindLoopStatement(preLoopLabel, postWhileLabel, () => {
this.walk(node.whileSuite); this.walk(node.whileSuite);
}); });
this._addAntecedent(preLoopLabel, this._currentFlowNode); this._addAntecedent(preLoopLabel, this._currentFlowNode);
this._currentFlowNode = constExprValue === true ? this._currentFlowNode =
Binder._unreachableFlowNode : this._finishFlowLabel(elseLabel); constExprValue === true ? Binder._unreachableFlowNode : this._finishFlowLabel(elseLabel);
if (node.elseSuite) { if (node.elseSuite) {
this.walk(node.elseSuite); this.walk(node.elseSuite);
} }
@ -737,8 +809,7 @@ export class Binder extends ParseTreeWalker {
node: node.name, node: node.name,
isConstant: isConstantName(node.name.value), isConstant: isConstantName(node.name.value),
path: this._fileInfo.filePath, path: this._fileInfo.filePath,
range: convertOffsetsToRange(node.name.start, TextRange.getEnd(node.name), range: convertOffsetsToRange(node.name.start, TextRange.getEnd(node.name), this._fileInfo.lines)
this._fileInfo.lines)
}; };
symbol.addDeclaration(declaration); symbol.addDeclaration(declaration);
} }
@ -757,9 +828,7 @@ export class Binder extends ParseTreeWalker {
visitRaise(node: RaiseNode): boolean { visitRaise(node: RaiseNode): boolean {
if (!node.typeExpression && this._nestedExceptDepth === 0) { if (!node.typeExpression && this._nestedExceptDepth === 0) {
this._addError( this._addError(`Raise requires parameter(s) when used outside of except clause `, node);
`Raise requires parameter(s) when used outside of except clause `,
node);
} }
if (node.typeExpression) { if (node.typeExpression) {
@ -893,8 +962,7 @@ export class Binder extends ParseTreeWalker {
antecedent: this._currentFlowNode, antecedent: this._currentFlowNode,
preFinallyGate preFinallyGate
}; };
this._currentFlowNode = isAfterElseAndExceptsReachable ? this._currentFlowNode = isAfterElseAndExceptsReachable ? postFinallyNode : Binder._unreachableFlowNode;
postFinallyNode : Binder._unreachableFlowNode;
} }
return false; return false;
@ -916,33 +984,34 @@ export class Binder extends ParseTreeWalker {
const unescapedResult = StringTokenUtils.getUnescapedString(stringNode.token); const unescapedResult = StringTokenUtils.getUnescapedString(stringNode.token);
unescapedResult.unescapeErrors.forEach((error: StringTokenUtils.UnescapeError) => { unescapedResult.unescapeErrors.forEach((error: StringTokenUtils.UnescapeError) => {
const start = stringNode.token.start + stringNode.token.prefixLength + const start =
stringNode.token.quoteMarkLength + error.offset; stringNode.token.start +
stringNode.token.prefixLength +
stringNode.token.quoteMarkLength +
error.offset;
const textRange = { start, length: error.length }; const textRange = { start, length: error.length };
if (error.errorType === StringTokenUtils.UnescapeErrorType.InvalidEscapeSequence) { if (error.errorType === StringTokenUtils.UnescapeErrorType.InvalidEscapeSequence) {
this._addDiagnostic( this._addDiagnostic(
this._fileInfo.diagnosticSettings.reportInvalidStringEscapeSequence, this._fileInfo.diagnosticSettings.reportInvalidStringEscapeSequence,
DiagnosticRule.reportInvalidStringEscapeSequence, DiagnosticRule.reportInvalidStringEscapeSequence,
'Unsupported escape sequence in string literal', textRange); 'Unsupported escape sequence in string literal',
} else if (error.errorType === textRange
StringTokenUtils.UnescapeErrorType.EscapeWithinFormatExpression) { );
} else if (error.errorType === StringTokenUtils.UnescapeErrorType.EscapeWithinFormatExpression) {
this._addError( this._addError(
'Escape sequence (backslash) not allowed in expression portion of f-string', 'Escape sequence (backslash) not allowed in expression portion of f-string',
textRange); textRange
} else if (error.errorType === );
StringTokenUtils.UnescapeErrorType.SingleCloseBraceWithinFormatLiteral) { } else if (
error.errorType === StringTokenUtils.UnescapeErrorType.SingleCloseBraceWithinFormatLiteral
) {
this._addError( this._addError(
'Single close brace not allowed within f-string literal; use double close brace', 'Single close brace not allowed within f-string literal; use double close brace',
textRange); textRange
} else if (error.errorType === );
StringTokenUtils.UnescapeErrorType.UnterminatedFormatExpression) { } else if (error.errorType === StringTokenUtils.UnescapeErrorType.UnterminatedFormatExpression) {
this._addError('Unterminated expression in f-string; missing close brace', textRange);
this._addError(
'Unterminated expression in f-string; missing close brace',
textRange);
} }
}); });
} }
@ -959,14 +1028,14 @@ export class Binder extends ParseTreeWalker {
// Is the binding inconsistent? // Is the binding inconsistent?
if (this._notLocalBindings.get(nameValue) === NameBindingType.Nonlocal) { if (this._notLocalBindings.get(nameValue) === NameBindingType.Nonlocal) {
this._addError(`'${ nameValue }' was already declared nonlocal`, name); this._addError(`'${nameValue}' was already declared nonlocal`, name);
} }
const valueWithScope = this._currentScope.lookUpSymbolRecursive(nameValue); const valueWithScope = this._currentScope.lookUpSymbolRecursive(nameValue);
// Was the name already assigned within this scope before it was declared global? // Was the name already assigned within this scope before it was declared global?
if (valueWithScope && valueWithScope.scope === this._currentScope) { if (valueWithScope && valueWithScope.scope === this._currentScope) {
this._addError(`'${ nameValue }' is assigned before global declaration`, name); this._addError(`'${nameValue}' is assigned before global declaration`, name);
} }
// Add it to the global scope if it's not already added. // Add it to the global scope if it's not already added.
@ -991,16 +1060,16 @@ export class Binder extends ParseTreeWalker {
// Is the binding inconsistent? // Is the binding inconsistent?
if (this._notLocalBindings.get(nameValue) === NameBindingType.Global) { if (this._notLocalBindings.get(nameValue) === NameBindingType.Global) {
this._addError(`'${ nameValue }' was already declared global`, name); this._addError(`'${nameValue}' was already declared global`, name);
} }
const valueWithScope = this._currentScope.lookUpSymbolRecursive(nameValue); const valueWithScope = this._currentScope.lookUpSymbolRecursive(nameValue);
// Was the name already assigned within this scope before it was declared nonlocal? // Was the name already assigned within this scope before it was declared nonlocal?
if (valueWithScope && valueWithScope.scope === this._currentScope) { if (valueWithScope && valueWithScope.scope === this._currentScope) {
this._addError(`'${ nameValue }' is assigned before nonlocal declaration`, name); this._addError(`'${nameValue}' is assigned before nonlocal declaration`, name);
} else if (!valueWithScope || valueWithScope.scope === globalScope) { } else if (!valueWithScope || valueWithScope.scope === globalScope) {
this._addError(`No binding for nonlocal '${ nameValue }' found`, name); this._addError(`No binding for nonlocal '${nameValue}' found`, name);
} }
this._notLocalBindings.set(nameValue, NameBindingType.Nonlocal); this._notLocalBindings.set(nameValue, NameBindingType.Nonlocal);
@ -1043,11 +1112,11 @@ export class Binder extends ParseTreeWalker {
// we'll build a single declaration that describes the combined actions // we'll build a single declaration that describes the combined actions
// of both import statements, thus reflecting the behavior of the // of both import statements, thus reflecting the behavior of the
// python module loader. // python module loader.
const existingDecl = symbol.getDeclarations().find( const existingDecl = symbol
decl => decl.type === DeclarationType.Alias && .getDeclarations()
decl.firstNamePart === firstNamePartValue); .find(decl => decl.type === DeclarationType.Alias && decl.firstNamePart === firstNamePartValue);
const newDecl: AliasDeclaration = existingDecl as AliasDeclaration || { const newDecl: AliasDeclaration = (existingDecl as AliasDeclaration) || {
type: DeclarationType.Alias, type: DeclarationType.Alias,
node, node,
path: '', path: '',
@ -1072,9 +1141,9 @@ export class Binder extends ParseTreeWalker {
const namePartValue = node.module.nameParts[i].value; const namePartValue = node.module.nameParts[i].value;
// Is there an existing loader action for this name? // Is there an existing loader action for this name?
let loaderActions = curLoaderActions.implicitImports ? let loaderActions = curLoaderActions.implicitImports
curLoaderActions.implicitImports.get(namePartValue) : ? curLoaderActions.implicitImports.get(namePartValue)
undefined; : undefined;
if (!loaderActions) { if (!loaderActions) {
// Allocate a new loader action. // Allocate a new loader action.
loaderActions = { loaderActions = {
@ -1367,8 +1436,7 @@ export class Binder extends ParseTreeWalker {
// if it's the same name as a symbol in an outer scope. If so, we'll // if it's the same name as a symbol in an outer scope. If so, we'll
// create an alias node in the control flow graph. // create an alias node in the control flow graph.
for (const addedSymbol of addedSymbols) { for (const addedSymbol of addedSymbols) {
const aliasSymbol = this._currentScope.parent!. const aliasSymbol = this._currentScope.parent!.lookUpSymbol(addedSymbol[0]);
lookUpSymbol(addedSymbol[0]);
if (aliasSymbol) { if (aliasSymbol) {
this._createAssignmentAliasFlowNode(addedSymbol[1].id, aliasSymbol.id); this._createAssignmentAliasFlowNode(addedSymbol[1].id, aliasSymbol.id);
} }
@ -1411,10 +1479,11 @@ export class Binder extends ParseTreeWalker {
const expr = firstDecl.node.parent.rightExpression; const expr = firstDecl.node.parent.rightExpression;
if (expr.nodeType === ParseNodeType.List) { if (expr.nodeType === ParseNodeType.List) {
expr.entries.forEach(listEntryNode => { expr.entries.forEach(listEntryNode => {
if (listEntryNode.nodeType === ParseNodeType.StringList && if (
listEntryNode.strings.length === 1 && listEntryNode.nodeType === ParseNodeType.StringList &&
listEntryNode.strings[0].nodeType === ParseNodeType.String) { listEntryNode.strings.length === 1 &&
listEntryNode.strings[0].nodeType === ParseNodeType.String
) {
const entryName = listEntryNode.strings[0].value; const entryName = listEntryNode.strings[0].value;
if (lookupInfo.symbolTable.get(entryName)) { if (lookupInfo.symbolTable.get(entryName)) {
namesToImport.push(entryName); namesToImport.push(entryName);
@ -1527,26 +1596,29 @@ export class Binder extends ParseTreeWalker {
this._currentFalseTarget = savedFalseTarget; this._currentFalseTarget = savedFalseTarget;
if (!this._isLogicalExpression(node)) { if (!this._isLogicalExpression(node)) {
this._addAntecedent(trueTarget, this._addAntecedent(
this._createFlowConditional(FlowFlags.TrueCondition, trueTarget,
this._currentFlowNode, node)); this._createFlowConditional(FlowFlags.TrueCondition, this._currentFlowNode, node)
this._addAntecedent(falseTarget, );
this._createFlowConditional(FlowFlags.FalseCondition, this._addAntecedent(
this._currentFlowNode, node)); falseTarget,
this._createFlowConditional(FlowFlags.FalseCondition, this._currentFlowNode, node)
);
} }
} }
private _createFlowConditional(flags: FlowFlags, antecedent: FlowNode, private _createFlowConditional(flags: FlowFlags, antecedent: FlowNode, expression: ExpressionNode): FlowNode {
expression: ExpressionNode): FlowNode {
if (antecedent.flags & FlowFlags.Unreachable) { if (antecedent.flags & FlowFlags.Unreachable) {
return antecedent; return antecedent;
} }
const staticValue = StaticExpressions.evaluateStaticBoolLikeExpression( const staticValue = StaticExpressions.evaluateStaticBoolLikeExpression(
expression, this._fileInfo.executionEnvironment); expression,
if (staticValue === true && (flags & FlowFlags.FalseCondition) || this._fileInfo.executionEnvironment
staticValue === false && (flags & FlowFlags.TrueCondition)) { );
if (
(staticValue === true && flags & FlowFlags.FalseCondition) ||
(staticValue === false && flags & FlowFlags.TrueCondition)
) {
return Binder._unreachableFlowNode; return Binder._unreachableFlowNode;
} }
@ -1580,17 +1652,14 @@ export class Binder extends ParseTreeWalker {
} }
case ParseNodeType.BinaryOperation: { case ParseNodeType.BinaryOperation: {
return expression.operator === OperatorType.And || return expression.operator === OperatorType.And || expression.operator === OperatorType.Or;
expression.operator === OperatorType.Or;
} }
} }
return false; return false;
} }
private _isNarrowingExpression(expression: ExpressionNode, private _isNarrowingExpression(expression: ExpressionNode, expressionList: NarrowingExpressionNode[]): boolean {
expressionList: NarrowingExpressionNode[]): boolean {
switch (expression.nodeType) { switch (expression.nodeType) {
case ParseNodeType.Name: case ParseNodeType.Name:
case ParseNodeType.MemberAccess: { case ParseNodeType.MemberAccess: {
@ -1612,30 +1681,34 @@ export class Binder extends ParseTreeWalker {
} }
case ParseNodeType.BinaryOperation: { case ParseNodeType.BinaryOperation: {
const isOrIsNotOperator = expression.operator === OperatorType.Is || const isOrIsNotOperator =
expression.operator === OperatorType.IsNot; expression.operator === OperatorType.Is || expression.operator === OperatorType.IsNot;
const equalsOrNotEqualsOperator = expression.operator === OperatorType.Equals || const equalsOrNotEqualsOperator =
expression.operator === OperatorType.NotEquals; expression.operator === OperatorType.Equals || expression.operator === OperatorType.NotEquals;
if (isOrIsNotOperator || equalsOrNotEqualsOperator) { if (isOrIsNotOperator || equalsOrNotEqualsOperator) {
// Look for "X is None", "X is not None", "X == None", "X != None". // Look for "X is None", "X is not None", "X == None", "X != None".
// These are commonly-used patterns used in control flow. // These are commonly-used patterns used in control flow.
if (expression.rightExpression.nodeType === ParseNodeType.Constant && if (
expression.rightExpression.constType === KeywordType.None) { expression.rightExpression.nodeType === ParseNodeType.Constant &&
expression.rightExpression.constType === KeywordType.None
) {
return this._isNarrowingExpression(expression.leftExpression, expressionList); return this._isNarrowingExpression(expression.leftExpression, expressionList);
} }
// Look for "type(X) is Y" or "type(X) is not Y". // Look for "type(X) is Y" or "type(X) is not Y".
if (isOrIsNotOperator && if (
isOrIsNotOperator &&
expression.leftExpression.nodeType === ParseNodeType.Call && expression.leftExpression.nodeType === ParseNodeType.Call &&
expression.leftExpression.leftExpression.nodeType === ParseNodeType.Name && expression.leftExpression.leftExpression.nodeType === ParseNodeType.Name &&
expression.leftExpression.leftExpression.value === 'type' && expression.leftExpression.leftExpression.value === 'type' &&
expression.leftExpression.arguments.length === 1 && expression.leftExpression.arguments.length === 1 &&
expression.leftExpression.arguments[0].argumentCategory === ArgumentCategory.Simple) { expression.leftExpression.arguments[0].argumentCategory === ArgumentCategory.Simple
) {
return this._isNarrowingExpression( return this._isNarrowingExpression(
expression.leftExpression.arguments[0].valueExpression, expressionList); expression.leftExpression.arguments[0].valueExpression,
expressionList
);
} }
} }
@ -1643,8 +1716,10 @@ export class Binder extends ParseTreeWalker {
} }
case ParseNodeType.UnaryOperation: { case ParseNodeType.UnaryOperation: {
return expression.operator === OperatorType.Not && return (
this._isNarrowingExpression(expression.expression, expressionList); expression.operator === OperatorType.Not &&
this._isNarrowingExpression(expression.expression, expressionList)
);
} }
case ParseNodeType.AugmentedAssignment: { case ParseNodeType.AugmentedAssignment: {
@ -1652,21 +1727,23 @@ export class Binder extends ParseTreeWalker {
} }
case ParseNodeType.Call: { case ParseNodeType.Call: {
if (expression.leftExpression.nodeType === ParseNodeType.Name && if (
(expression.leftExpression.value === 'isinstance' || expression.leftExpression.nodeType === ParseNodeType.Name &&
expression.leftExpression.value === 'issubclass') && (expression.leftExpression.value === 'isinstance' ||
expression.arguments.length === 2) { expression.leftExpression.value === 'issubclass') &&
expression.arguments.length === 2
) {
return this._isNarrowingExpression(expression.arguments[0].valueExpression, expressionList); return this._isNarrowingExpression(expression.arguments[0].valueExpression, expressionList);
} }
if (expression.leftExpression.nodeType === ParseNodeType.Name &&
expression.leftExpression.value === 'callable' &&
expression.arguments.length === 1) {
if (
expression.leftExpression.nodeType === ParseNodeType.Name &&
expression.leftExpression.value === 'callable' &&
expression.arguments.length === 1
) {
return this._isNarrowingExpression(expression.arguments[0].valueExpression, expressionList); return this._isNarrowingExpression(expression.arguments[0].valueExpression, expressionList);
} }
} }
} }
return false; return false;
@ -1805,13 +1882,13 @@ export class Binder extends ParseTreeWalker {
} }
private _addExceptTargets(flowNode: FlowNode) { private _addExceptTargets(flowNode: FlowNode) {
// If there are any except targets, then we're in a try block, and we // If there are any except targets, then we're in a try block, and we
// have to assume that an exception can be raised after every assignment. // have to assume that an exception can be raised after every assignment.
if (this._currentExceptTargets) { if (this._currentExceptTargets) {
this._currentExceptTargets.forEach(label => { this._currentExceptTargets.forEach(label => {
this._addAntecedent(label, flowNode); this._addAntecedent(label, flowNode);
}); });
} }
} }
private _bindLoopStatement(preLoopLabel: FlowLabel, postLoopLabel: FlowLabel, callback: () => void) { private _bindLoopStatement(preLoopLabel: FlowLabel, postLoopLabel: FlowLabel, callback: () => void) {
@ -1840,8 +1917,7 @@ export class Binder extends ParseTreeWalker {
// Don't overwrite an existing symbol. // Don't overwrite an existing symbol.
let symbol = scope.lookUpSymbol(name); let symbol = scope.lookUpSymbol(name);
if (!symbol) { if (!symbol) {
symbol = scope.addSymbol(name, symbol = scope.addSymbol(name, SymbolFlags.InitiallyUnbound | SymbolFlags.ClassMember);
SymbolFlags.InitiallyUnbound | SymbolFlags.ClassMember);
if (this._fileInfo.isStubFile && isPrivateOrProtectedName(name)) { if (this._fileInfo.isStubFile && isPrivateOrProtectedName(name)) {
symbol.setIsExternallyHidden(); symbol.setIsExternallyHidden();
@ -1891,9 +1967,11 @@ export class Binder extends ParseTreeWalker {
} }
} }
private _addBuiltInSymbolToCurrentScope(nameValue: string, private _addBuiltInSymbolToCurrentScope(
node: ModuleNode | ClassNode | FunctionNode, type: IntrinsicType) { nameValue: string,
node: ModuleNode | ClassNode | FunctionNode,
type: IntrinsicType
) {
const symbol = this._addSymbolToCurrentScope(nameValue, false); const symbol = this._addSymbolToCurrentScope(nameValue, false);
if (symbol) { if (symbol) {
symbol.addDeclaration({ symbol.addDeclaration({
@ -1937,15 +2015,13 @@ export class Binder extends ParseTreeWalker {
return symbol; return symbol;
} }
private _createNewScope(scopeType: ScopeType, parentScope: Scope | undefined, private _createNewScope(scopeType: ScopeType, parentScope: Scope | undefined, callback: () => void) {
callback: () => void) {
const prevScope = this._currentScope; const prevScope = this._currentScope;
this._currentScope = new Scope(scopeType, parentScope); this._currentScope = new Scope(scopeType, parentScope);
// If this scope is an execution scope, allocate a new reference map. // If this scope is an execution scope, allocate a new reference map.
const isExecutionScope = scopeType === ScopeType.Builtin || scopeType === ScopeType.Module || const isExecutionScope =
scopeType === ScopeType.Function; scopeType === ScopeType.Builtin || scopeType === ScopeType.Module || scopeType === ScopeType.Function;
const prevReferenceMap = this._currentExecutionScopeReferenceMap; const prevReferenceMap = this._currentExecutionScopeReferenceMap;
if (isExecutionScope) { if (isExecutionScope) {
@ -1988,10 +2064,8 @@ export class Binder extends ParseTreeWalker {
let symbol = memberAccessInfo.classScope.lookUpSymbol(name.value); let symbol = memberAccessInfo.classScope.lookUpSymbol(name.value);
if (!symbol) { if (!symbol) {
symbol = memberAccessInfo.classScope.addSymbol(name.value, symbol = memberAccessInfo.classScope.addSymbol(name.value, SymbolFlags.InitiallyUnbound);
SymbolFlags.InitiallyUnbound); const honorPrivateNaming = this._fileInfo.diagnosticSettings.reportPrivateUsage !== 'none';
const honorPrivateNaming =
this._fileInfo.diagnosticSettings.reportPrivateUsage !== 'none';
if (isPrivateOrProtectedName(name.value) && honorPrivateNaming) { if (isPrivateOrProtectedName(name.value) && honorPrivateNaming) {
symbol.setIsPrivateMember(); symbol.setIsPrivateMember();
} }
@ -2009,9 +2083,11 @@ export class Binder extends ParseTreeWalker {
isConstant: isConstantName(name.value), isConstant: isConstantName(name.value),
inferredTypeSource: source, inferredTypeSource: source,
path: this._fileInfo.filePath, path: this._fileInfo.filePath,
range: convertOffsetsToRange(target.memberName.start, range: convertOffsetsToRange(
target.memberName.start,
target.memberName.start + target.memberName.length, target.memberName.start + target.memberName.length,
this._fileInfo.lines) this._fileInfo.lines
)
}; };
symbol.addDeclaration(declaration); symbol.addDeclaration(declaration);
} }
@ -2059,8 +2135,7 @@ export class Binder extends ParseTreeWalker {
isConstant: isConstantName(name.value), isConstant: isConstantName(name.value),
isFinal: finalInfo.isFinal, isFinal: finalInfo.isFinal,
path: this._fileInfo.filePath, path: this._fileInfo.filePath,
typeAnnotationNode: finalInfo.isFinal ? typeAnnotationNode: finalInfo.isFinal ? finalInfo.finalTypeNode : typeAnnotation,
finalInfo.finalTypeNode : typeAnnotation,
range: convertOffsetsToRange(name.start, TextRange.getEnd(name), this._fileInfo.lines) range: convertOffsetsToRange(name.start, TextRange.getEnd(name), this._fileInfo.lines)
}; };
symbolWithScope.symbol.addDeclaration(declaration); symbolWithScope.symbol.addDeclaration(declaration);
@ -2085,10 +2160,8 @@ export class Binder extends ParseTreeWalker {
let symbol = memberAccessInfo.classScope.lookUpSymbol(name.value); let symbol = memberAccessInfo.classScope.lookUpSymbol(name.value);
if (!symbol) { if (!symbol) {
symbol = memberAccessInfo.classScope.addSymbol(name.value, symbol = memberAccessInfo.classScope.addSymbol(name.value, SymbolFlags.InitiallyUnbound);
SymbolFlags.InitiallyUnbound); const honorPrivateNaming = this._fileInfo.diagnosticSettings.reportPrivateUsage !== 'none';
const honorPrivateNaming =
this._fileInfo.diagnosticSettings.reportPrivateUsage !== 'none';
if (isPrivateOrProtectedName(name.value) && honorPrivateNaming) { if (isPrivateOrProtectedName(name.value) && honorPrivateNaming) {
symbol.setIsPrivateMember(); symbol.setIsPrivateMember();
} }
@ -2107,11 +2180,12 @@ export class Binder extends ParseTreeWalker {
isConstant: isConstantName(name.value), isConstant: isConstantName(name.value),
isFinal: finalInfo.isFinal, isFinal: finalInfo.isFinal,
path: this._fileInfo.filePath, path: this._fileInfo.filePath,
typeAnnotationNode: finalInfo.isFinal ? typeAnnotationNode: finalInfo.isFinal ? finalInfo.finalTypeNode : typeAnnotation,
finalInfo.finalTypeNode : typeAnnotation, range: convertOffsetsToRange(
range: convertOffsetsToRange(target.memberName.start, target.memberName.start,
target.memberName.start + target.memberName.length, target.memberName.start + target.memberName.length,
this._fileInfo.lines) this._fileInfo.lines
)
}; };
symbol.addDeclaration(declaration); symbol.addDeclaration(declaration);
@ -2122,9 +2196,7 @@ export class Binder extends ParseTreeWalker {
} }
if (!declarationHandled) { if (!declarationHandled) {
this._addError( this._addError(`Type annotation not supported for this type of expression`, typeAnnotation);
`Type annotation not supported for this type of expression`,
typeAnnotation);
} }
} }
@ -2145,15 +2217,14 @@ export class Binder extends ParseTreeWalker {
isFinal = true; isFinal = true;
} }
} else if (typeAnnotation.nodeType === ParseNodeType.MemberAccess) { } else if (typeAnnotation.nodeType === ParseNodeType.MemberAccess) {
if (typeAnnotation.leftExpression.nodeType === ParseNodeType.Name && if (
typeAnnotation.leftExpression.nodeType === ParseNodeType.Name &&
typeAnnotation.leftExpression.value === 'typing' && typeAnnotation.leftExpression.value === 'typing' &&
typeAnnotation.memberName.value === 'Final') { typeAnnotation.memberName.value === 'Final'
) {
isFinal = true; isFinal = true;
} }
} else if (typeAnnotation.nodeType === ParseNodeType.Index && } else if (typeAnnotation.nodeType === ParseNodeType.Index && typeAnnotation.items.items.length === 1) {
typeAnnotation.items.items.length === 1) {
// Recursively call to see if the base expression is "Final". // Recursively call to see if the base expression is "Final".
const finalInfo = this._isAnnotationFinal(typeAnnotation.baseExpression); const finalInfo = this._isAnnotationFinal(typeAnnotation.baseExpression);
if (finalInfo.isFinal) { if (finalInfo.isFinal) {
@ -2249,9 +2320,9 @@ export class Binder extends ParseTreeWalker {
private _addImplicitImportsToLoaderActions(importResult: ImportResult, loaderActions: ModuleLoaderActions) { private _addImplicitImportsToLoaderActions(importResult: ImportResult, loaderActions: ModuleLoaderActions) {
importResult.implicitImports.forEach(implicitImport => { importResult.implicitImports.forEach(implicitImport => {
const existingLoaderAction = loaderActions.implicitImports ? const existingLoaderAction = loaderActions.implicitImports
loaderActions.implicitImports.get(implicitImport.name) : ? loaderActions.implicitImports.get(implicitImport.name)
undefined; : undefined;
if (existingLoaderAction) { if (existingLoaderAction) {
existingLoaderAction.path = implicitImport.path; existingLoaderAction.path = implicitImport.path;
} else { } else {
@ -2291,15 +2362,15 @@ export class Binder extends ParseTreeWalker {
const assignedNameNode = annotationNode.valueExpression; const assignedNameNode = annotationNode.valueExpression;
const specialTypes: { [name: string]: boolean } = { const specialTypes: { [name: string]: boolean } = {
'Tuple': true, Tuple: true,
'Generic': true, Generic: true,
'Protocol': true, Protocol: true,
'Callable': true, Callable: true,
'Type': true, Type: true,
'ClassVar': true, ClassVar: true,
'Final': true, Final: true,
'Literal': true, Literal: true,
'TypedDict': true TypedDict: true
}; };
const assignedName = assignedNameNode.value; const assignedName = assignedNameNode.value;
@ -2314,8 +2385,11 @@ export class Binder extends ParseTreeWalker {
type: DeclarationType.SpecialBuiltInClass, type: DeclarationType.SpecialBuiltInClass,
node: annotationNode, node: annotationNode,
path: this._fileInfo.filePath, path: this._fileInfo.filePath,
range: convertOffsetsToRange(annotationNode.start, range: convertOffsetsToRange(
TextRange.getEnd(annotationNode), this._fileInfo.lines) annotationNode.start,
TextRange.getEnd(annotationNode),
this._fileInfo.lines
)
}); });
} }
return true; return true;
@ -2348,13 +2422,11 @@ export class Binder extends ParseTreeWalker {
const functionNode = ParseTreeUtils.getEnclosingFunction(node); const functionNode = ParseTreeUtils.getEnclosingFunction(node);
if (!functionNode) { if (!functionNode) {
this._addError( this._addError(`'yield' not allowed outside of a function`, node);
`'yield' not allowed outside of a function`, node);
} else if (functionNode.isAsync && node.nodeType === ParseNodeType.YieldFrom) { } else if (functionNode.isAsync && node.nodeType === ParseNodeType.YieldFrom) {
// PEP 525 indicates that 'yield from' is not allowed in an // PEP 525 indicates that 'yield from' is not allowed in an
// async function. // async function.
this._addError( this._addError(`'yield from' not allowed in an async function`, node);
`'yield from' not allowed in an async function`, node);
} }
if (this._targetFunctionDeclaration) { if (this._targetFunctionDeclaration) {
@ -2372,9 +2444,7 @@ export class Binder extends ParseTreeWalker {
AnalyzerNodeInfo.setFlowNode(node, this._currentFlowNode); AnalyzerNodeInfo.setFlowNode(node, this._currentFlowNode);
} }
private _addDiagnostic(diagLevel: DiagnosticLevel, rule: string, private _addDiagnostic(diagLevel: DiagnosticLevel, rule: string, message: string, textRange: TextRange) {
message: string, textRange: TextRange) {
if (diagLevel === 'error') { if (diagLevel === 'error') {
const diagnostic = this._addError(message, textRange); const diagnostic = this._addError(message, textRange);
diagnostic.setRule(rule); diagnostic.setRule(rule);
@ -2388,8 +2458,7 @@ export class Binder extends ParseTreeWalker {
} }
private _addUnusedCode(textRange: TextRange) { private _addUnusedCode(textRange: TextRange) {
return this._fileInfo.diagnosticSink.addUnusedCodeWithTextRange( return this._fileInfo.diagnosticSink.addUnusedCodeWithTextRange('Code is unreachable', textRange);
'Code is unreachable', textRange);
} }
private _addError(message: string, textRange: TextRange) { private _addError(message: string, textRange: TextRange) {

517
server/src/analyzer/checker.ts
View File

@ -1,29 +1,64 @@
/* /*
* checker.ts * checker.ts
* Copyright (c) Microsoft Corporation. * Copyright (c) Microsoft Corporation.
* Licensed under the MIT license. * Licensed under the MIT license.
* Author: Eric Traut * Author: Eric Traut
* *
* A parse tree walker that performs static type checking for * A parse tree walker that performs static type checking for
* a source file. Most of its work is performed by the type * a source file. Most of its work is performed by the type
* evaluator, but this module touches every node in the file * evaluator, but this module touches every node in the file
* to ensure that all statements and expressions are evaluated * to ensure that all statements and expressions are evaluated
* and checked. It also performs some additional checks that * and checked. It also performs some additional checks that
* cannot (or should not be) performed lazily. * cannot (or should not be) performed lazily.
*/ */
import { DiagnosticLevel } from '../common/configOptions'; import { DiagnosticLevel } from '../common/configOptions';
import { assert } from '../common/debug'; import { assert } from '../common/debug';
import { Diagnostic, DiagnosticAddendum } from '../common/diagnostic'; import { Diagnostic, DiagnosticAddendum } from '../common/diagnostic';
import { DiagnosticRule } from '../common/diagnosticRules'; import { DiagnosticRule } from '../common/diagnosticRules';
import { TextRange } from '../common/textRange'; import { TextRange } from '../common/textRange';
import { AssertNode, AssignmentExpressionNode, AssignmentNode, AugmentedAssignmentNode, import {
BinaryOperationNode, CallNode, ClassNode, DelNode, ErrorNode, ExceptNode, AssertNode,
FormatStringNode, ForNode, FunctionNode, IfNode, ImportAsNode, ImportFromAsNode, ImportFromNode, AssignmentExpressionNode,
IndexNode, LambdaNode, ListComprehensionNode, MemberAccessNode, ModuleNode, NameNode, AssignmentNode,
ParameterCategory, ParseNode, ParseNodeType, RaiseNode, ReturnNode, SliceNode, StringListNode, AugmentedAssignmentNode,
SuiteNode, TernaryNode, TupleNode, TypeAnnotationNode, UnaryOperationNode, UnpackNode, BinaryOperationNode,
WhileNode, WithNode, YieldFromNode, YieldNode } from '../parser/parseNodes'; CallNode,
ClassNode,
DelNode,
ErrorNode,
ExceptNode,
FormatStringNode,
ForNode,
FunctionNode,
IfNode,
ImportAsNode,
ImportFromAsNode,
ImportFromNode,
IndexNode,
LambdaNode,
ListComprehensionNode,
MemberAccessNode,
ModuleNode,
NameNode,
ParameterCategory,
ParseNode,
ParseNodeType,
RaiseNode,
ReturnNode,
SliceNode,
StringListNode,
SuiteNode,
TernaryNode,
TupleNode,
TypeAnnotationNode,
UnaryOperationNode,
UnpackNode,
WhileNode,
WithNode,
YieldFromNode,
YieldNode
} from '../parser/parseNodes';
import { AnalyzerFileInfo } from './analyzerFileInfo'; import { AnalyzerFileInfo } from './analyzerFileInfo';
import * as AnalyzerNodeInfo from './analyzerNodeInfo'; import * as AnalyzerNodeInfo from './analyzerNodeInfo';
import { Declaration, DeclarationType } from './declaration'; import { Declaration, DeclarationType } from './declaration';
@ -36,11 +71,31 @@ import { Symbol } from './symbol';
import * as SymbolNameUtils from './symbolNameUtils'; import * as SymbolNameUtils from './symbolNameUtils';
import { getLastTypedDeclaredForSymbol, isFinalVariable } from './symbolUtils'; import { getLastTypedDeclaredForSymbol, isFinalVariable } from './symbolUtils';
import { TypeEvaluator } from './typeEvaluator'; import { TypeEvaluator } from './typeEvaluator';
import { ClassType, combineTypes, FunctionType, isAnyOrUnknown, isNoneOrNever, isTypeSame, import {
NoneType, ObjectType, Type, TypeCategory, UnknownType } from './types'; ClassType,
import { containsUnknown, derivesFromClassRecursive, doForSubtypes, combineTypes,
getDeclaredGeneratorReturnType, getDeclaredGeneratorYieldType, getSymbolFromBaseClasses, FunctionType,
isNoReturnType, isProperty, specializeType, transformTypeObjectToClass } from './typeUtils'; isAnyOrUnknown,
isNoneOrNever,
isTypeSame,
NoneType,
ObjectType,
Type,
TypeCategory,
UnknownType
} from './types';
import {
containsUnknown,
derivesFromClassRecursive,
doForSubtypes,
getDeclaredGeneratorReturnType,
getDeclaredGeneratorYieldType,
getSymbolFromBaseClasses,
isNoReturnType,
isProperty,
specializeType,
transformTypeObjectToClass
} from './typeUtils';
export class Checker extends ParseTreeWalker { export class Checker extends ParseTreeWalker {
private readonly _moduleNode: ModuleNode; private readonly _moduleNode: ModuleNode;
@ -52,7 +107,6 @@ export class Checker extends ParseTreeWalker {
private _scopedNodes: AnalyzerNodeInfo.ScopedNode[] = []; private _scopedNodes: AnalyzerNodeInfo.ScopedNode[] = [];
constructor(node: ModuleNode, evaluator: TypeEvaluator) { constructor(node: ModuleNode, evaluator: TypeEvaluator) {
super(); super();
this._moduleNode = node; this._moduleNode = node;
@ -87,9 +141,9 @@ export class Checker extends ParseTreeWalker {
if (classTypeResult) { if (classTypeResult) {
this._validateClassMethods(classTypeResult.classType); this._validateClassMethods(classTypeResult.classType);
this._validateFinalMemberOverrides(classTypeResult.classType); this._validateFinalMemberOverrides(classTypeResult.classType);
if (ClassType.isTypedDictClass(classTypeResult.classType)) { if (ClassType.isTypedDictClass(classTypeResult.classType)) {
this._validateTypedDictClassSuite(node.suite); this._validateTypedDictClassSuite(node.suite);
} }
@ -113,16 +167,18 @@ export class Checker extends ParseTreeWalker {
this._evaluator.addDiagnostic( this._evaluator.addDiagnostic(
this._fileInfo.diagnosticSettings.reportUnknownParameterType, this._fileInfo.diagnosticSettings.reportUnknownParameterType,
DiagnosticRule.reportUnknownParameterType, DiagnosticRule.reportUnknownParameterType,
`Type of parameter '${ param.name.value }' is unknown`, `Type of parameter '${param.name.value}' is unknown`,
param.name); param.name
);
} else if (containsUnknown(paramType)) { } else if (containsUnknown(paramType)) {
const diagAddendum = new DiagnosticAddendum(); const diagAddendum = new DiagnosticAddendum();
diagAddendum.addMessage(`Parameter type is '${ this._evaluator.printType(paramType) }'`); diagAddendum.addMessage(`Parameter type is '${this._evaluator.printType(paramType)}'`);
this._evaluator.addDiagnostic( this._evaluator.addDiagnostic(
this._fileInfo.diagnosticSettings.reportUnknownParameterType, this._fileInfo.diagnosticSettings.reportUnknownParameterType,
DiagnosticRule.reportUnknownParameterType, DiagnosticRule.reportUnknownParameterType,
`Type of parameter '${ param.name.value }' is partially unknown` + diagAddendum.getString(), `Type of parameter '${param.name.value}' is partially unknown` + diagAddendum.getString(),
param.name); param.name
);
} }
} }
}); });
@ -180,15 +236,17 @@ export class Checker extends ParseTreeWalker {
this._evaluator.addDiagnostic( this._evaluator.addDiagnostic(
this._fileInfo.diagnosticSettings.reportUnknownLambdaType, this._fileInfo.diagnosticSettings.reportUnknownLambdaType,
DiagnosticRule.reportUnknownLambdaType, DiagnosticRule.reportUnknownLambdaType,
`Type of '${ param.name.value }' is unknown`, `Type of '${param.name.value}' is unknown`,
param.name); param.name
);
} else if (containsUnknown(paramType)) { } else if (containsUnknown(paramType)) {
this._evaluator.addDiagnostic( this._evaluator.addDiagnostic(
this._fileInfo.diagnosticSettings.reportUnknownLambdaType, this._fileInfo.diagnosticSettings.reportUnknownLambdaType,
DiagnosticRule.reportUnknownLambdaType, DiagnosticRule.reportUnknownLambdaType,
`Type of '${ param.name.value }', ` + `Type of '${param.name.value}', ` +
`'${ this._evaluator.printType(paramType) }', is partially unknown`, `'${this._evaluator.printType(paramType)}', is partially unknown`,
param.name); param.name
);
} }
} }
} }
@ -200,13 +258,16 @@ export class Checker extends ParseTreeWalker {
this._evaluator.addDiagnostic( this._evaluator.addDiagnostic(
this._fileInfo.diagnosticSettings.reportUnknownLambdaType, this._fileInfo.diagnosticSettings.reportUnknownLambdaType,
DiagnosticRule.reportUnknownLambdaType, DiagnosticRule.reportUnknownLambdaType,
`Type of lambda expression is unknown`, node.expression); `Type of lambda expression is unknown`,
node.expression
);
} else if (containsUnknown(returnType)) { } else if (containsUnknown(returnType)) {
this._evaluator.addDiagnostic( this._evaluator.addDiagnostic(
this._fileInfo.diagnosticSettings.reportUnknownLambdaType, this._fileInfo.diagnosticSettings.reportUnknownLambdaType,
DiagnosticRule.reportUnknownLambdaType, DiagnosticRule.reportUnknownLambdaType,
`Type of lambda expression, '${ this._evaluator.printType(returnType) }', is partially unknown`, `Type of lambda expression, '${this._evaluator.printType(returnType)}', is partially unknown`,
node.expression); node.expression
);
} }
} }
@ -225,7 +286,8 @@ export class Checker extends ParseTreeWalker {
this._fileInfo.diagnosticSettings.reportCallInDefaultInitializer, this._fileInfo.diagnosticSettings.reportCallInDefaultInitializer,
DiagnosticRule.reportCallInDefaultInitializer, DiagnosticRule.reportCallInDefaultInitializer,
`Function calls within default value initializer are not permitted`, `Function calls within default value initializer are not permitted`,
node); node
);
} }
return true; return true;
@ -263,9 +325,9 @@ export class Checker extends ParseTreeWalker {
let returnType: Type; let returnType: Type;
const enclosingFunctionNode = ParseTreeUtils.getEnclosingFunction(node); const enclosingFunctionNode = ParseTreeUtils.getEnclosingFunction(node);
const declaredReturnType = enclosingFunctionNode ? const declaredReturnType = enclosingFunctionNode
this._evaluator.getFunctionDeclaredReturnType(enclosingFunctionNode) : ? this._evaluator.getFunctionDeclaredReturnType(enclosingFunctionNode)
undefined; : undefined;
if (node.returnExpression) { if (node.returnExpression) {
returnType = this._evaluator.getType(node.returnExpression) || UnknownType.create(); returnType = this._evaluator.getType(node.returnExpression) || UnknownType.create();
@ -279,7 +341,8 @@ export class Checker extends ParseTreeWalker {
if (isNoReturnType(declaredReturnType)) { if (isNoReturnType(declaredReturnType)) {
this._evaluator.addError( this._evaluator.addError(
`Function with declared return type 'NoReturn' cannot include a return statement`, `Function with declared return type 'NoReturn' cannot include a return statement`,
node); node
);
} else { } else {
const diagAddendum = new DiagnosticAddendum(); const diagAddendum = new DiagnosticAddendum();
@ -288,10 +351,11 @@ export class Checker extends ParseTreeWalker {
const specializedDeclaredType = specializeType(declaredReturnType, undefined); const specializedDeclaredType = specializeType(declaredReturnType, undefined);
if (!this._evaluator.canAssignType(specializedDeclaredType, returnType, diagAddendum)) { if (!this._evaluator.canAssignType(specializedDeclaredType, returnType, diagAddendum)) {
this._evaluator.addError( this._evaluator.addError(
`Expression of type '${ this._evaluator.printType(returnType) }' cannot be assigned ` + `Expression of type '${this._evaluator.printType(returnType)}' cannot be assigned ` +
`to return type '${ this._evaluator.printType(specializedDeclaredType) }'` + `to return type '${this._evaluator.printType(specializedDeclaredType)}'` +
diagAddendum.getString(), diagAddendum.getString(),
node.returnExpression ? node.returnExpression : node); node.returnExpression ? node.returnExpression : node
);
} }
} }
} }
@ -300,13 +364,16 @@ export class Checker extends ParseTreeWalker {
this._evaluator.addDiagnostic( this._evaluator.addDiagnostic(
this._fileInfo.diagnosticSettings.reportUnknownVariableType, this._fileInfo.diagnosticSettings.reportUnknownVariableType,
DiagnosticRule.reportUnknownVariableType, DiagnosticRule.reportUnknownVariableType,
`Return type is unknown`, node.returnExpression!); `Return type is unknown`,
node.returnExpression!
);
} else if (containsUnknown(returnType)) { } else if (containsUnknown(returnType)) {
this._evaluator.addDiagnostic( this._evaluator.addDiagnostic(
this._fileInfo.diagnosticSettings.reportUnknownVariableType, this._fileInfo.diagnosticSettings.reportUnknownVariableType,
DiagnosticRule.reportUnknownVariableType, DiagnosticRule.reportUnknownVariableType,
`Return type, '${ this._evaluator.printType(returnType) }', is partially unknown`, `Return type, '${this._evaluator.printType(returnType)}', is partially unknown`,
node.returnExpression!); node.returnExpression!
);
} }
} }
@ -314,8 +381,7 @@ export class Checker extends ParseTreeWalker {
} }
visitYield(node: YieldNode) { visitYield(node: YieldNode) {
const yieldType = node.expression ? const yieldType = node.expression ? this._evaluator.getType(node.expression) : NoneType.create();
this._evaluator.getType(node.expression) : NoneType.create();
// Wrap the yield type in an Iterator. // Wrap the yield type in an Iterator.
let adjYieldType = yieldType; let adjYieldType = yieldType;
@ -354,14 +420,20 @@ export class Checker extends ParseTreeWalker {
if (!isAnyOrUnknown(subtype)) { if (!isAnyOrUnknown(subtype)) {
if (subtype.category === TypeCategory.Class) { if (subtype.category === TypeCategory.Class) {
if (!derivesFromClassRecursive(subtype, baseExceptionType)) { if (!derivesFromClassRecursive(subtype, baseExceptionType)) {
diagAddendum.addMessage(`'${ this._evaluator.printType(subtype) }' does not derive from BaseException`); diagAddendum.addMessage(
`'${this._evaluator.printType(subtype)}' does not derive from BaseException`
);
} }
} else if (subtype.category === TypeCategory.Object) { } else if (subtype.category === TypeCategory.Object) {
if (!derivesFromClassRecursive(subtype.classType, baseExceptionType)) { if (!derivesFromClassRecursive(subtype.classType, baseExceptionType)) {
diagAddendum.addMessage(`'${ this._evaluator.printType(subtype) }' does not derive from BaseException`); diagAddendum.addMessage(
`'${this._evaluator.printType(subtype)}' does not derive from BaseException`
);
} }
} else { } else {
diagAddendum.addMessage(`'${ this._evaluator.printType(subtype) }' does not derive from BaseException`); diagAddendum.addMessage(
`'${this._evaluator.printType(subtype)}' does not derive from BaseException`
);
} }
} }
@ -371,7 +443,8 @@ export class Checker extends ParseTreeWalker {
if (diagAddendum.getMessageCount() > 0) { if (diagAddendum.getMessageCount() > 0) {
this._evaluator.addError( this._evaluator.addError(
`Expected exception class or object` + diagAddendum.getString(), `Expected exception class or object` + diagAddendum.getString(),
node.typeExpression); node.typeExpression
);
} }
} }
} }
@ -387,10 +460,14 @@ export class Checker extends ParseTreeWalker {
if (!isAnyOrUnknown(subtype) && !isNoneOrNever(subtype)) { if (!isAnyOrUnknown(subtype) && !isNoneOrNever(subtype)) {
if (subtype.category === TypeCategory.Object) { if (subtype.category === TypeCategory.Object) {
if (!derivesFromClassRecursive(subtype.classType, baseExceptionType)) { if (!derivesFromClassRecursive(subtype.classType, baseExceptionType)) {
diagAddendum.addMessage(`'${ this._evaluator.printType(subtype) }' does not derive from BaseException`); diagAddendum.addMessage(
`'${this._evaluator.printType(subtype)}' does not derive from BaseException`
);
} }
} else { } else {
diagAddendum.addMessage(`'${ this._evaluator.printType(subtype) }' does not derive from BaseException`); diagAddendum.addMessage(
`'${this._evaluator.printType(subtype)}' does not derive from BaseException`
);
} }
} }
@ -400,7 +477,8 @@ export class Checker extends ParseTreeWalker {
if (diagAddendum.getMessageCount() > 0) { if (diagAddendum.getMessageCount() > 0) {
this._evaluator.addError( this._evaluator.addError(
`Expected exception object or None` + diagAddendum.getString(), `Expected exception object or None` + diagAddendum.getString(),
node.valueExpression); node.valueExpression
);
} }
} }
} }
@ -430,10 +508,13 @@ export class Checker extends ParseTreeWalker {
if (type && type.category === TypeCategory.Object) { if (type && type.category === TypeCategory.Object) {
if (ClassType.isBuiltIn(type.classType, 'Tuple') && type.classType.typeArguments) { if (ClassType.isBuiltIn(type.classType, 'Tuple') && type.classType.typeArguments) {
if (type.classType.typeArguments.length > 0) { if (type.classType.typeArguments.length > 0) {
this._evaluator.addDiagnosticForTextRange(this._fileInfo, this._evaluator.addDiagnosticForTextRange(
this._fileInfo,
this._fileInfo.diagnosticSettings.reportAssertAlwaysTrue, this._fileInfo.diagnosticSettings.reportAssertAlwaysTrue,
DiagnosticRule.reportAssertAlwaysTrue, DiagnosticRule.reportAssertAlwaysTrue,
`Assert expression always evaluates to true`, node.testExpression); `Assert expression always evaluates to true`,
node.testExpression
);
} }
} }
} }
@ -501,10 +582,13 @@ export class Checker extends ParseTreeWalker {
} }
if (node.strings.length > 1) { if (node.strings.length > 1) {
this._evaluator.addDiagnosticForTextRange(this._fileInfo, this._evaluator.addDiagnosticForTextRange(
this._fileInfo,
this._fileInfo.diagnosticSettings.reportImplicitStringConcatenation, this._fileInfo.diagnosticSettings.reportImplicitStringConcatenation,
DiagnosticRule.reportImplicitStringConcatenation, DiagnosticRule.reportImplicitStringConcatenation,
`Implicit string concatenation not allowed`, node); `Implicit string concatenation not allowed`,
node
);
} }
return true; return true;
@ -591,12 +675,12 @@ export class Checker extends ParseTreeWalker {
} else if (exceptionType.category === TypeCategory.Class) { } else if (exceptionType.category === TypeCategory.Class) {
if (!derivesFromBaseException(exceptionType)) { if (!derivesFromBaseException(exceptionType)) {
diagAddendum.addMessage( diagAddendum.addMessage(
`'${this._evaluator.printType(exceptionType)}' does not derive from BaseException`); `'${this._evaluator.printType(exceptionType)}' does not derive from BaseException`
);
} }
resultingExceptionType = ObjectType.create(exceptionType); resultingExceptionType = ObjectType.create(exceptionType);
} else if (exceptionType.category === TypeCategory.Object) { } else if (exceptionType.category === TypeCategory.Object) {
const iterableType = this._evaluator.getTypeFromIterable( const iterableType = this._evaluator.getTypeFromIterable(exceptionType, false, errorNode, false);
exceptionType, false, errorNode, false);
resultingExceptionType = doForSubtypes(iterableType, subtype => { resultingExceptionType = doForSubtypes(iterableType, subtype => {
if (isAnyOrUnknown(subtype)) { if (isAnyOrUnknown(subtype)) {
@ -607,23 +691,25 @@ export class Checker extends ParseTreeWalker {
if (transformedSubtype.category === TypeCategory.Class) { if (transformedSubtype.category === TypeCategory.Class) {
if (!derivesFromBaseException(transformedSubtype)) { if (!derivesFromBaseException(transformedSubtype)) {
diagAddendum.addMessage( diagAddendum.addMessage(
`'${this._evaluator.printType(exceptionType)}' does not derive from BaseException`); `'${this._evaluator.printType(exceptionType)}' does not derive from BaseException`
);
} }
return ObjectType.create(transformedSubtype); return ObjectType.create(transformedSubtype);
} }
diagAddendum.addMessage( diagAddendum.addMessage(
`'${this._evaluator.printType(exceptionType)}' does not derive from BaseException`); `'${this._evaluator.printType(exceptionType)}' does not derive from BaseException`
);
return UnknownType.create(); return UnknownType.create();
}); });
} }
if (diagAddendum.getMessageCount() > 0) { if (diagAddendum.getMessageCount() > 0) {
this._evaluator.addError( this._evaluator.addError(
`'${this._evaluator.printType(exceptionType)}' is not valid exception class` + `'${this._evaluator.printType(exceptionType)}' is not valid exception class` + diagAddendum.getString(),
diagAddendum.getString(), errorNode
errorNode); );
} }
return resultingExceptionType || UnknownType.create(); return resultingExceptionType || UnknownType.create();
@ -660,20 +746,14 @@ export class Checker extends ParseTreeWalker {
decls.forEach(decl => { decls.forEach(decl => {
if (isFinalVariableDeclaration(decl)) { if (isFinalVariableDeclaration(decl)) {
if (sawFinal) { if (sawFinal) {
this._evaluator.addError( this._evaluator.addError(`'${name}' was previously declared as Final`, decl.node);
`'${ name }' was previously declared as Final`,
decl.node
);
} }
sawFinal = true; sawFinal = true;
} }
if (decl.type === DeclarationType.Variable && decl.inferredTypeSource) { if (decl.type === DeclarationType.Variable && decl.inferredTypeSource) {
if (sawAssignment) { if (sawAssignment) {
this._evaluator.addError( this._evaluator.addError(`'${name}' is declared Final and can be assigned only once`, decl.node);
`'${ name }' is declared Final and can be assigned only once`,
decl.node
);
} }
sawAssignment = true; sawAssignment = true;
} }
@ -683,10 +763,7 @@ export class Checker extends ParseTreeWalker {
if (!sawAssignment && !this._fileInfo.isStubFile) { if (!sawAssignment && !this._fileInfo.isStubFile) {
const firstDecl = decls.find(decl => decl.type === DeclarationType.Variable && decl.isFinal); const firstDecl = decls.find(decl => decl.type === DeclarationType.Variable && decl.isFinal);
if (firstDecl) { if (firstDecl) {
this._evaluator.addError( this._evaluator.addError(`'${name}' is declared Final, but value is not assigned`, firstDecl.node);
`'${ name }' is declared Final, but value is not assigned`,
firstDecl.node
);
} }
} }
} }
@ -729,9 +806,7 @@ export class Checker extends ParseTreeWalker {
const addPrimaryDeclInfo = (diag?: Diagnostic) => { const addPrimaryDeclInfo = (diag?: Diagnostic) => {
if (diag) { if (diag) {
let primaryDeclNode: ParseNode | undefined; let primaryDeclNode: ParseNode | undefined;
if (primaryDecl.type === DeclarationType.Function || if (primaryDecl.type === DeclarationType.Function || primaryDecl.type === DeclarationType.Class) {
primaryDecl.type === DeclarationType.Class) {
primaryDeclNode = primaryDecl.node.name; primaryDeclNode = primaryDecl.node.name;
} else if (primaryDecl.type === DeclarationType.Variable) { } else if (primaryDecl.type === DeclarationType.Variable) {
if (primaryDecl.node.nodeType === ParseNodeType.Name) { if (primaryDecl.node.nodeType === ParseNodeType.Name) {
@ -744,8 +819,7 @@ export class Checker extends ParseTreeWalker {
} }
if (primaryDeclNode) { if (primaryDeclNode) {
diag.addRelatedInfo(`See ${ primaryDeclType }declaration`, diag.addRelatedInfo(`See ${primaryDeclType}declaration`, primaryDecl.path, primaryDecl.range);
primaryDecl.path, primaryDecl.range);
} }
} }
}; };
@ -753,14 +827,13 @@ export class Checker extends ParseTreeWalker {
for (const otherDecl of otherDecls) { for (const otherDecl of otherDecls) {
if (otherDecl.type === DeclarationType.Class) { if (otherDecl.type === DeclarationType.Class) {
const diag = this._evaluator.addError( const diag = this._evaluator.addError(
`Class declaration '${ name }' is obscured by a ${ primaryDeclType }` + `Class declaration '${name}' is obscured by a ${primaryDeclType}` + `declaration of the same name`,
`declaration of the same name`,
otherDecl.node.name otherDecl.node.name
); );
addPrimaryDeclInfo(diag); addPrimaryDeclInfo(diag);
} else if (otherDecl.type === DeclarationType.Function) { } else if (otherDecl.type === DeclarationType.Function) {
const diag = this._evaluator.addError( const diag = this._evaluator.addError(
`Function declaration '${ name }' is obscured by a ${ primaryDeclType }` + `Function declaration '${name}' is obscured by a ${primaryDeclType}` +
`declaration of the same name`, `declaration of the same name`,
otherDecl.node.name otherDecl.node.name
); );
@ -768,8 +841,7 @@ export class Checker extends ParseTreeWalker {
} else if (otherDecl.type === DeclarationType.Parameter) { } else if (otherDecl.type === DeclarationType.Parameter) {
if (otherDecl.node.name) { if (otherDecl.node.name) {
const diag = this._evaluator.addError( const diag = this._evaluator.addError(
`Parameter '${ name }' is obscured by a ${ primaryDeclType }` + `Parameter '${name}' is obscured by a ${primaryDeclType}` + `declaration of the same name`,
`declaration of the same name`,
otherDecl.node.name otherDecl.node.name
); );
addPrimaryDeclInfo(diag); addPrimaryDeclInfo(diag);
@ -792,8 +864,8 @@ export class Checker extends ParseTreeWalker {
if (!duplicateIsOk) { if (!duplicateIsOk) {
const diag = this._evaluator.addError( const diag = this._evaluator.addError(
`Declared type for '${ name }' is obscured by an ` + `Declared type for '${name}' is obscured by an ` +
`incompatible ${ primaryDeclType }declaration`, `incompatible ${primaryDeclType}declaration`,
otherDecl.node otherDecl.node
); );
addPrimaryDeclInfo(diag); addPrimaryDeclInfo(diag);
@ -802,7 +874,7 @@ export class Checker extends ParseTreeWalker {
} else if (primaryType && !isProperty(primaryType)) { } else if (primaryType && !isProperty(primaryType)) {
if (primaryDecl.type === DeclarationType.Function || primaryDecl.type === DeclarationType.Class) { if (primaryDecl.type === DeclarationType.Function || primaryDecl.type === DeclarationType.Class) {
const diag = this._evaluator.addError( const diag = this._evaluator.addError(
`Declared ${ primaryDeclType }already exists for '${ name }'`, `Declared ${primaryDeclType}already exists for '${name}'`,
otherDecl.node otherDecl.node
); );
addPrimaryDeclInfo(diag); addPrimaryDeclInfo(diag);
@ -830,8 +902,7 @@ export class Checker extends ParseTreeWalker {
const decls = symbol.getDeclarations(); const decls = symbol.getDeclarations();
decls.forEach(decl => { decls.forEach(decl => {
this._conditionallyReportUnusedDeclaration(decl, this._conditionallyReportUnusedDeclaration(decl, this._isSymbolPrivate(name, scopeType));
this._isSymbolPrivate(name, scopeType));
}); });
} }
@ -856,12 +927,17 @@ export class Checker extends ParseTreeWalker {
const textRange: TextRange = { start: nameParts[0].start, length: nameParts[0].length }; const textRange: TextRange = { start: nameParts[0].start, length: nameParts[0].length };
TextRange.extend(textRange, nameParts[nameParts.length - 1]); TextRange.extend(textRange, nameParts[nameParts.length - 1]);
this._fileInfo.diagnosticSink.addUnusedCodeWithTextRange( this._fileInfo.diagnosticSink.addUnusedCodeWithTextRange(
`'${ multipartName }' is not accessed`, textRange); `'${multipartName}' is not accessed`,
textRange
);
this._evaluator.addDiagnosticForTextRange(this._fileInfo, this._evaluator.addDiagnosticForTextRange(
this._fileInfo,
this._fileInfo.diagnosticSettings.reportUnusedImport, this._fileInfo.diagnosticSettings.reportUnusedImport,
DiagnosticRule.reportUnusedImport, DiagnosticRule.reportUnusedImport,
`Import '${ multipartName }' is not accessed`, textRange); `Import '${multipartName}' is not accessed`,
textRange
);
return; return;
} }
} }
@ -870,16 +946,17 @@ export class Checker extends ParseTreeWalker {
// unused imports. Don't report these because they're in generated // unused imports. Don't report these because they're in generated
// files that shouldn't be edited. // files that shouldn't be edited.
const importFrom = decl.node.parent as ImportFromNode; const importFrom = decl.node.parent as ImportFromNode;
if (importFrom.module.nameParts.length === 0 || if (
importFrom.module.nameParts[0].value !== '__future__' && importFrom.module.nameParts.length === 0 ||
!this._fileInfo.filePath.endsWith('_pb2.py')) { (importFrom.module.nameParts[0].value !== '__future__' &&
!this._fileInfo.filePath.endsWith('_pb2.py'))
) {
nameNode = decl.node.alias || decl.node.name; nameNode = decl.node.alias || decl.node.name;
} }
} }
if (nameNode) { if (nameNode) {
message = `Import '${ nameNode.value }' is not accessed`; message = `Import '${nameNode.value}' is not accessed`;
} }
break; break;
@ -892,7 +969,7 @@ export class Checker extends ParseTreeWalker {
if (decl.node.nodeType === ParseNodeType.Name) { if (decl.node.nodeType === ParseNodeType.Name) {
nameNode = decl.node; nameNode = decl.node;
rule = DiagnosticRule.reportUnusedVariable; rule = DiagnosticRule.reportUnusedVariable;
message = `Variable '${ nameNode.value }' is not accessed`; message = `Variable '${nameNode.value}' is not accessed`;
} }
break; break;
@ -903,7 +980,7 @@ export class Checker extends ParseTreeWalker {
diagnosticLevel = this._fileInfo.diagnosticSettings.reportUnusedClass; diagnosticLevel = this._fileInfo.diagnosticSettings.reportUnusedClass;
nameNode = decl.node.name; nameNode = decl.node.name;
rule = DiagnosticRule.reportUnusedClass; rule = DiagnosticRule.reportUnusedClass;
message = `Class '${ nameNode.value }' is not accessed`; message = `Class '${nameNode.value}' is not accessed`;
break; break;
case DeclarationType.Function: case DeclarationType.Function:
@ -913,7 +990,7 @@ export class Checker extends ParseTreeWalker {
diagnosticLevel = this._fileInfo.diagnosticSettings.reportUnusedFunction; diagnosticLevel = this._fileInfo.diagnosticSettings.reportUnusedFunction;
nameNode = decl.node.name; nameNode = decl.node.name;
rule = DiagnosticRule.reportUnusedFunction; rule = DiagnosticRule.reportUnusedFunction;
message = `Function '${ nameNode.value }' is not accessed`; message = `Function '${nameNode.value}' is not accessed`;
break; break;
default: default:
@ -921,20 +998,19 @@ export class Checker extends ParseTreeWalker {
} }
if (nameNode && rule !== undefined && message) { if (nameNode && rule !== undefined && message) {
this._fileInfo.diagnosticSink.addUnusedCodeWithTextRange( this._fileInfo.diagnosticSink.addUnusedCodeWithTextRange(`'${nameNode.value}' is not accessed`, nameNode);
`'${ nameNode.value }' is not accessed`, nameNode); this._evaluator.addDiagnostic(diagnosticLevel, rule, message, nameNode);
this._evaluator.addDiagnostic(
diagnosticLevel, rule, message, nameNode);
} }
} }
// Validates that a call to isinstance or issubclass are necessary. This is a // Validates that a call to isinstance or issubclass are necessary. This is a
// common source of programming errors. // common source of programming errors.
private _validateIsInstanceCallNecessary(node: CallNode) { private _validateIsInstanceCallNecessary(node: CallNode) {
if (node.leftExpression.nodeType !== ParseNodeType.Name || if (
(node.leftExpression.value !== 'isinstance' && node.leftExpression.nodeType !== ParseNodeType.Name ||
node.leftExpression.value !== 'issubclass') || (node.leftExpression.value !== 'isinstance' && node.leftExpression.value !== 'issubclass') ||
node.arguments.length !== 2) { node.arguments.length !== 2
) {
return; return;
} }
@ -990,15 +1066,17 @@ export class Checker extends ParseTreeWalker {
// According to PEP 544, protocol classes cannot be used as the right-hand // According to PEP 544, protocol classes cannot be used as the right-hand
// argument to isinstance or issubclass. // argument to isinstance or issubclass.
if (classTypeList.some(type => ClassType.isProtocol(type))) { if (classTypeList.some(type => ClassType.isProtocol(type))) {
this._evaluator.addError(`Protocol class cannot be used in ${ callName } call`, this._evaluator.addError(
node.arguments[1].valueExpression); `Protocol class cannot be used in ${callName} call`,
node.arguments[1].valueExpression
);
} }
const finalizeFilteredTypeList = (types: Type[]): Type => { const finalizeFilteredTypeList = (types: Type[]): Type => {
return combineTypes(types); return combineTypes(types);
}; };
const filterType = (varType: ClassType): (ObjectType[] | ClassType[]) => { const filterType = (varType: ClassType): ObjectType[] | ClassType[] => {
const filteredTypes: ClassType[] = []; const filteredTypes: ClassType[] = [];
for (const filterType of classTypeList) { for (const filterType of classTypeList) {
@ -1067,16 +1145,18 @@ export class Checker extends ParseTreeWalker {
this._evaluator.addDiagnostic( this._evaluator.addDiagnostic(
this._fileInfo.diagnosticSettings.reportUnnecessaryIsInstance, this._fileInfo.diagnosticSettings.reportUnnecessaryIsInstance,
DiagnosticRule.reportUnnecessaryIsInstance, DiagnosticRule.reportUnnecessaryIsInstance,
`Unnecessary ${ callName } call: '${ this._evaluator.printType(arg0Type) }' ` + `Unnecessary ${callName} call: '${this._evaluator.printType(arg0Type)}' ` +
`is never ${ callType } of '${ this._evaluator.printType(getTestType()) }'`, `is never ${callType} of '${this._evaluator.printType(getTestType())}'`,
node); node
);
} else if (isTypeSame(filteredType, arg0Type)) { } else if (isTypeSame(filteredType, arg0Type)) {
this._evaluator.addDiagnostic( this._evaluator.addDiagnostic(
this._fileInfo.diagnosticSettings.reportUnnecessaryIsInstance, this._fileInfo.diagnosticSettings.reportUnnecessaryIsInstance,
DiagnosticRule.reportUnnecessaryIsInstance, DiagnosticRule.reportUnnecessaryIsInstance,
`Unnecessary ${ callName } call: '${ this._evaluator.printType(arg0Type) }' ` + `Unnecessary ${callName} call: '${this._evaluator.printType(arg0Type)}' ` +
`is always ${ callType } of '${ this._evaluator.printType(getTestType()) }'`, `is always ${callType} of '${this._evaluator.printType(getTestType())}'`,
node); node
);
} }
} }
@ -1123,8 +1203,8 @@ export class Checker extends ParseTreeWalker {
const declarations = this._evaluator.getDeclarationsForNameNode(node); const declarations = this._evaluator.getDeclarationsForNameNode(node);
let primaryDeclaration = declarations && declarations.length > 0 ? let primaryDeclaration =
declarations[declarations.length - 1] : undefined; declarations && declarations.length > 0 ? declarations[declarations.length - 1] : undefined;
if (!primaryDeclaration || primaryDeclaration.node === node) { if (!primaryDeclaration || primaryDeclaration.node === node) {
return; return;
} }
@ -1136,17 +1216,17 @@ export class Checker extends ParseTreeWalker {
let classOrModuleNode: ClassNode | ModuleNode | undefined; let classOrModuleNode: ClassNode | ModuleNode | undefined;
if (primaryDeclaration.node) { if (primaryDeclaration.node) {
classOrModuleNode = ParseTreeUtils.getEnclosingClassOrModule( classOrModuleNode = ParseTreeUtils.getEnclosingClassOrModule(primaryDeclaration.node);
primaryDeclaration.node);
} }
// If this is the name of a class, find the module or class that contains it rather // If this is the name of a class, find the module or class that contains it rather
// than constraining the use of the class name within the class itself. // than constraining the use of the class name within the class itself.
if (primaryDeclaration.node && if (
primaryDeclaration.node.parent && primaryDeclaration.node &&
primaryDeclaration.node.parent === classOrModuleNode && primaryDeclaration.node.parent &&
classOrModuleNode.nodeType === ParseNodeType.Class) { primaryDeclaration.node.parent === classOrModuleNode &&
classOrModuleNode.nodeType === ParseNodeType.Class
) {
classOrModuleNode = ParseTreeUtils.getEnclosingClassOrModule(classOrModuleNode); classOrModuleNode = ParseTreeUtils.getEnclosingClassOrModule(classOrModuleNode);
} }
@ -1166,12 +1246,16 @@ export class Checker extends ParseTreeWalker {
// If the referencing class is a subclass of the declaring class, it's // If the referencing class is a subclass of the declaring class, it's
// allowed to access a protected name. // allowed to access a protected name.
if (enclosingClassTypeInfo && if (
enclosingClassTypeInfo.decoratedType.category === TypeCategory.Class) { enclosingClassTypeInfo &&
enclosingClassTypeInfo.decoratedType.category === TypeCategory.Class
if (derivesFromClassRecursive(enclosingClassTypeInfo.decoratedType, ) {
declClassTypeInfo.decoratedType)) { if (
derivesFromClassRecursive(
enclosingClassTypeInfo.decoratedType,
declClassTypeInfo.decoratedType
)
) {
return; return;
} }
} }
@ -1185,17 +1269,18 @@ export class Checker extends ParseTreeWalker {
this._evaluator.addDiagnostic( this._evaluator.addDiagnostic(
this._fileInfo.diagnosticSettings.reportPrivateUsage, this._fileInfo.diagnosticSettings.reportPrivateUsage,
DiagnosticRule.reportPrivateUsage, DiagnosticRule.reportPrivateUsage,
`'${ nameValue }' is protected and used outside of a derived class`, `'${nameValue}' is protected and used outside of a derived class`,
node); node
);
} else { } else {
const scopeName = classOrModuleNode.nodeType === ParseNodeType.Class ? const scopeName = classOrModuleNode.nodeType === ParseNodeType.Class ? 'class' : 'module';
'class' : 'module';
this._evaluator.addDiagnostic( this._evaluator.addDiagnostic(
this._fileInfo.diagnosticSettings.reportPrivateUsage, this._fileInfo.diagnosticSettings.reportPrivateUsage,
DiagnosticRule.reportPrivateUsage, DiagnosticRule.reportPrivateUsage,
`'${ nameValue }' is private and used outside of the ${ scopeName } in which it is declared`, `'${nameValue}' is private and used outside of the ${scopeName} in which it is declared`,
node); node
);
} }
} }
} }
@ -1205,20 +1290,19 @@ export class Checker extends ParseTreeWalker {
// strings, and "pass" statements or ellipses. // strings, and "pass" statements or ellipses.
private _validateTypedDictClassSuite(suiteNode: SuiteNode) { private _validateTypedDictClassSuite(suiteNode: SuiteNode) {
const emitBadStatementError = (node: ParseNode) => { const emitBadStatementError = (node: ParseNode) => {
this._evaluator.addError( this._evaluator.addError(`TypedDict classes can contain only type annotations`, node);
`TypedDict classes can contain only type annotations`,
node);
}; };
suiteNode.statements.forEach(statement => { suiteNode.statements.forEach(statement => {
if (!AnalyzerNodeInfo.isCodeUnreachable(statement)) { if (!AnalyzerNodeInfo.isCodeUnreachable(statement)) {
if (statement.nodeType === ParseNodeType.StatementList) { if (statement.nodeType === ParseNodeType.StatementList) {
for (const substatement of statement.statements) { for (const substatement of statement.statements) {
if (substatement.nodeType !== ParseNodeType.TypeAnnotation && if (
substatement.nodeType !== ParseNodeType.Ellipsis && substatement.nodeType !== ParseNodeType.TypeAnnotation &&
substatement.nodeType !== ParseNodeType.StringList && substatement.nodeType !== ParseNodeType.Ellipsis &&
substatement.nodeType !== ParseNodeType.Pass) { substatement.nodeType !== ParseNodeType.StringList &&
substatement.nodeType !== ParseNodeType.Pass
) {
emitBadStatementError(substatement); emitBadStatementError(substatement);
} }
} }
@ -1247,13 +1331,18 @@ export class Checker extends ParseTreeWalker {
this._evaluator.addDiagnostic( this._evaluator.addDiagnostic(
this._fileInfo.diagnosticSettings.reportUnknownVariableType, this._fileInfo.diagnosticSettings.reportUnknownVariableType,
DiagnosticRule.reportUnknownVariableType, DiagnosticRule.reportUnknownVariableType,
`Declared return type is unknown`, node.returnTypeAnnotation); `Declared return type is unknown`,
node.returnTypeAnnotation
);
} else if (containsUnknown(declaredReturnType)) { } else if (containsUnknown(declaredReturnType)) {
this._evaluator.addDiagnostic( this._evaluator.addDiagnostic(
this._fileInfo.diagnosticSettings.reportUnknownVariableType, this._fileInfo.diagnosticSettings.reportUnknownVariableType,
DiagnosticRule.reportUnknownVariableType, DiagnosticRule.reportUnknownVariableType,
`Declared return type, '${ this._evaluator.printType(declaredReturnType) }', is partially unknown`, `Declared return type, '${this._evaluator.printType(
node.returnTypeAnnotation); declaredReturnType
)}', is partially unknown`,
node.returnTypeAnnotation
);
} }
} }
@ -1273,7 +1362,8 @@ export class Checker extends ParseTreeWalker {
if (!ParseTreeUtils.isSuiteEmpty(node.suite)) { if (!ParseTreeUtils.isSuiteEmpty(node.suite)) {
this._evaluator.addError( this._evaluator.addError(
`Function with declared type of 'NoReturn' cannot return 'None'`, `Function with declared type of 'NoReturn' cannot return 'None'`,
node.returnTypeAnnotation); node.returnTypeAnnotation
);
} }
} else if (!FunctionType.isAbstractMethod(functionType)) { } else if (!FunctionType.isAbstractMethod(functionType)) {
// Make sure that the function doesn't implicitly return None if the declared // Make sure that the function doesn't implicitly return None if the declared
@ -1287,9 +1377,11 @@ export class Checker extends ParseTreeWalker {
// the return type matches. // the return type matches.
if (!ParseTreeUtils.isSuiteEmpty(node.suite)) { if (!ParseTreeUtils.isSuiteEmpty(node.suite)) {
this._evaluator.addError( this._evaluator.addError(
`Function with declared type of '${ this._evaluator.printType(declaredReturnType) }'` + `Function with declared type of '${this._evaluator.printType(declaredReturnType)}'` +
` must return value` + diagAddendum.getString(), ` must return value` +
node.returnTypeAnnotation); diagAddendum.getString(),
node.returnTypeAnnotation
);
} }
} }
} }
@ -1300,13 +1392,16 @@ export class Checker extends ParseTreeWalker {
this._evaluator.addDiagnostic( this._evaluator.addDiagnostic(
this._fileInfo.diagnosticSettings.reportUnknownParameterType, this._fileInfo.diagnosticSettings.reportUnknownParameterType,
DiagnosticRule.reportUnknownParameterType, DiagnosticRule.reportUnknownParameterType,
`Inferred return type is unknown`, node.name); `Inferred return type is unknown`,
node.name
);
} else if (containsUnknown(inferredReturnType)) { } else if (containsUnknown(inferredReturnType)) {
this._evaluator.addDiagnostic( this._evaluator.addDiagnostic(
this._fileInfo.diagnosticSettings.reportUnknownParameterType, this._fileInfo.diagnosticSettings.reportUnknownParameterType,
DiagnosticRule.reportUnknownParameterType, DiagnosticRule.reportUnknownParameterType,
`Return type '${ this._evaluator.printType(inferredReturnType) }' is partially unknown`, `Return type '${this._evaluator.printType(inferredReturnType)}' is partially unknown`,
node.name); node.name
);
} }
} }
} }
@ -1319,9 +1414,10 @@ export class Checker extends ParseTreeWalker {
if (parentSymbol && isFinalVariable(parentSymbol.symbol)) { if (parentSymbol && isFinalVariable(parentSymbol.symbol)) {
const decl = localSymbol.getDeclarations()[0]; const decl = localSymbol.getDeclarations()[0];
this._evaluator.addError( this._evaluator.addError(
`'${ name }' cannot be redeclared because parent class ` + `'${name}' cannot be redeclared because parent class ` +
`'${ parentSymbol.class.details.name }' declares it as Final`, `'${parentSymbol.class.details.name}' declares it as Final`,
decl.node); decl.node
);
} }
}); });
} }
@ -1346,21 +1442,22 @@ export class Checker extends ParseTreeWalker {
const baseClassAndSymbol = getSymbolFromBaseClasses(classType, name); const baseClassAndSymbol = getSymbolFromBaseClasses(classType, name);
if (baseClassAndSymbol) { if (baseClassAndSymbol) {
const typeOfBaseClassMethod = this._evaluator.getEffectiveTypeOfSymbol( const typeOfBaseClassMethod = this._evaluator.getEffectiveTypeOfSymbol(
baseClassAndSymbol.symbol); baseClassAndSymbol.symbol
);
const diagAddendum = new DiagnosticAddendum(); const diagAddendum = new DiagnosticAddendum();
if (!this._evaluator.canOverrideMethod(typeOfBaseClassMethod, typeOfSymbol, if (!this._evaluator.canOverrideMethod(typeOfBaseClassMethod, typeOfSymbol, diagAddendum)) {
diagAddendum)) {
const decl = getLastTypedDeclaredForSymbol(symbol); const decl = getLastTypedDeclaredForSymbol(symbol);
if (decl && decl.type === DeclarationType.Function) { if (decl && decl.type === DeclarationType.Function) {
const diag = this._evaluator.addDiagnostic( const diag = this._evaluator.addDiagnostic(
this._fileInfo.diagnosticSettings.reportIncompatibleMethodOverride, this._fileInfo.diagnosticSettings.reportIncompatibleMethodOverride,
DiagnosticRule.reportIncompatibleMethodOverride, DiagnosticRule.reportIncompatibleMethodOverride,
`Method '${ name }' overrides class '${ baseClassAndSymbol.class.details.name }' ` + `Method '${name}' overrides class '${baseClassAndSymbol.class.details.name}' ` +
`in an incompatible manner` + diagAddendum.getString(), decl.node.name); `in an incompatible manner` +
diagAddendum.getString(),
decl.node.name
);
const origDecl = getLastTypedDeclaredForSymbol( const origDecl = getLastTypedDeclaredForSymbol(baseClassAndSymbol.symbol);
baseClassAndSymbol.symbol);
if (diag && origDecl) { if (diag && origDecl) {
diag.addRelatedInfo('Overridden method', origDecl.path, origDecl.range); diag.addRelatedInfo('Overridden method', origDecl.path, origDecl.range);
} }
@ -1372,12 +1469,12 @@ export class Checker extends ParseTreeWalker {
const decl = getLastTypedDeclaredForSymbol(symbol); const decl = getLastTypedDeclaredForSymbol(symbol);
if (decl && decl.type === DeclarationType.Function) { if (decl && decl.type === DeclarationType.Function) {
const diag = this._evaluator.addError( const diag = this._evaluator.addError(
`Method '${ name }' cannot override final method defined ` + `Method '${name}' cannot override final method defined ` +
`in class '${ baseClassAndSymbol.class.details.name }'`, `in class '${baseClassAndSymbol.class.details.name}'`,
decl.node.name); decl.node.name
);
const origDecl = getLastTypedDeclaredForSymbol( const origDecl = getLastTypedDeclaredForSymbol(baseClassAndSymbol.symbol);
baseClassAndSymbol.symbol);
if (diag && origDecl) { if (diag && origDecl) {
diag.addRelatedInfo('Final method', origDecl.path, origDecl.range); diag.addRelatedInfo('Final method', origDecl.path, origDecl.range);
} }
@ -1395,24 +1492,27 @@ export class Checker extends ParseTreeWalker {
private _validateMethod(node: FunctionNode, functionType: FunctionType, classNode: ClassNode) { private _validateMethod(node: FunctionNode, functionType: FunctionType, classNode: ClassNode) {
if (node.name && node.name.value === '__new__') { if (node.name && node.name.value === '__new__') {
// __new__ overrides should have a "cls" parameter. // __new__ overrides should have a "cls" parameter.
if (node.parameters.length === 0 || !node.parameters[0].name || if (
(node.parameters[0].name.value !== 'cls' && node.parameters.length === 0 ||
node.parameters[0].name.value !== 'mcs')) { !node.parameters[0].name ||
(node.parameters[0].name.value !== 'cls' && node.parameters[0].name.value !== 'mcs')
) {
this._evaluator.addDiagnostic( this._evaluator.addDiagnostic(
this._fileInfo.diagnosticSettings.reportSelfClsParameterName, this._fileInfo.diagnosticSettings.reportSelfClsParameterName,
DiagnosticRule.reportSelfClsParameterName, DiagnosticRule.reportSelfClsParameterName,
`The __new__ override should take a 'cls' parameter`, `The __new__ override should take a 'cls' parameter`,
node.parameters.length > 0 ? node.parameters[0] : node.name); node.parameters.length > 0 ? node.parameters[0] : node.name
);
} }
} else if (node.name && node.name.value === '__init_subclass__') { } else if (node.name && node.name.value === '__init_subclass__') {
// __init_subclass__ overrides should have a "cls" parameter. // __init_subclass__ overrides should have a "cls" parameter.
if (node.parameters.length === 0 || !node.parameters[0].name || if (node.parameters.length === 0 || !node.parameters[0].name || node.parameters[0].name.value !== 'cls') {
node.parameters[0].name.value !== 'cls') {
this._evaluator.addDiagnostic( this._evaluator.addDiagnostic(
this._fileInfo.diagnosticSettings.reportSelfClsParameterName, this._fileInfo.diagnosticSettings.reportSelfClsParameterName,
DiagnosticRule.reportSelfClsParameterName, DiagnosticRule.reportSelfClsParameterName,
`The __init_subclass__ override should take a 'cls' parameter`, `The __init_subclass__ override should take a 'cls' parameter`,
node.parameters.length > 0 ? node.parameters[0] : node.name); node.parameters.length > 0 ? node.parameters[0] : node.name
);
} }
} else if (FunctionType.isStaticMethod(functionType)) { } else if (FunctionType.isStaticMethod(functionType)) {
// Static methods should not have "self" or "cls" parameters. // Static methods should not have "self" or "cls" parameters.
@ -1423,7 +1523,8 @@ export class Checker extends ParseTreeWalker {
this._fileInfo.diagnosticSettings.reportSelfClsParameterName, this._fileInfo.diagnosticSettings.reportSelfClsParameterName,
DiagnosticRule.reportSelfClsParameterName, DiagnosticRule.reportSelfClsParameterName,
`Static methods should not take a 'self' or 'cls' parameter`, `Static methods should not take a 'self' or 'cls' parameter`,
node.parameters[0].name); node.parameters[0].name
);
} }
} }
} else if (FunctionType.isClassMethod(functionType)) { } else if (FunctionType.isClassMethod(functionType)) {
@ -1440,7 +1541,8 @@ export class Checker extends ParseTreeWalker {
this._fileInfo.diagnosticSettings.reportSelfClsParameterName, this._fileInfo.diagnosticSettings.reportSelfClsParameterName,
DiagnosticRule.reportSelfClsParameterName, DiagnosticRule.reportSelfClsParameterName,
`Class methods should take a 'cls' parameter`, `Class methods should take a 'cls' parameter`,
node.parameters.length > 0 ? node.parameters[0] : node.name); node.parameters.length > 0 ? node.parameters[0] : node.name
);
} }
} }
} else { } else {
@ -1466,9 +1568,12 @@ export class Checker extends ParseTreeWalker {
if (paramName === 'cls') { if (paramName === 'cls') {
const classTypeInfo = this._evaluator.getTypeOfClass(classNode); const classTypeInfo = this._evaluator.getTypeOfClass(classNode);
const typeType = this._evaluator.getBuiltInType(classNode, 'type'); const typeType = this._evaluator.getBuiltInType(classNode, 'type');
if (typeType && typeType.category === TypeCategory.Class && if (
classTypeInfo && classTypeInfo.classType.category === TypeCategory.Class) { typeType &&
typeType.category === TypeCategory.Class &&
classTypeInfo &&
classTypeInfo.classType.category === TypeCategory.Class
) {
if (derivesFromClassRecursive(classTypeInfo.classType, typeType)) { if (derivesFromClassRecursive(classTypeInfo.classType, typeType)) {
isLegalMetaclassName = true; isLegalMetaclassName = true;
} }
@ -1480,7 +1585,8 @@ export class Checker extends ParseTreeWalker {
this._fileInfo.diagnosticSettings.reportSelfClsParameterName, this._fileInfo.diagnosticSettings.reportSelfClsParameterName,
DiagnosticRule.reportSelfClsParameterName, DiagnosticRule.reportSelfClsParameterName,
`Instance methods should take a 'self' parameter`, `Instance methods should take a 'self' parameter`,
node.parameters.length > 0 ? node.parameters[0] : node.name); node.parameters.length > 0 ? node.parameters[0] : node.name
);
} }
} }
} }
@ -1505,14 +1611,17 @@ export class Checker extends ParseTreeWalker {
if (isNoReturnType(declaredYieldType)) { if (isNoReturnType(declaredYieldType)) {
this._evaluator.addError( this._evaluator.addError(
`Function with declared return type 'NoReturn' cannot include a yield statement`, `Function with declared return type 'NoReturn' cannot include a yield statement`,
node); node
);
} else { } else {
const diagAddendum = new DiagnosticAddendum(); const diagAddendum = new DiagnosticAddendum();
if (!this._evaluator.canAssignType(declaredYieldType, adjustedYieldType, diagAddendum)) { if (!this._evaluator.canAssignType(declaredYieldType, adjustedYieldType, diagAddendum)) {
this._evaluator.addError( this._evaluator.addError(
`Expression of type '${ this._evaluator.printType(adjustedYieldType) }' cannot be assigned ` + `Expression of type '${this._evaluator.printType(adjustedYieldType)}' cannot be assigned ` +
`to yield type '${ this._evaluator.printType(declaredYieldType) }'` + diagAddendum.getString(), `to yield type '${this._evaluator.printType(declaredYieldType)}'` +
node.expression || node); diagAddendum.getString(),
node.expression || node
);
} }
} }
} }
@ -1536,8 +1645,9 @@ export class Checker extends ParseTreeWalker {
this._evaluator.addDiagnostic( this._evaluator.addDiagnostic(
this._fileInfo.diagnosticSettings.reportDuplicateImport, this._fileInfo.diagnosticSettings.reportDuplicateImport,
DiagnosticRule.reportDuplicateImport, DiagnosticRule.reportDuplicateImport,
`'${ importFromAs.name.value }' is imported more than once`, `'${importFromAs.name.value}' is imported more than once`,
importFromAs.name); importFromAs.name
);
} else { } else {
symbolMap.set(importFromAs.name.value, importFromAs); symbolMap.set(importFromAs.name.value, importFromAs);
} }
@ -1551,8 +1661,9 @@ export class Checker extends ParseTreeWalker {
this._evaluator.addDiagnostic( this._evaluator.addDiagnostic(
this._fileInfo.diagnosticSettings.reportDuplicateImport, this._fileInfo.diagnosticSettings.reportDuplicateImport,
DiagnosticRule.reportDuplicateImport, DiagnosticRule.reportDuplicateImport,
`'${ importStatement.moduleName }' is imported more than once`, `'${importStatement.moduleName}' is imported more than once`,
importStatement.subnode); importStatement.subnode
);
} else { } else {
importModuleMap.set(importStatement.moduleName, importStatement.subnode); importModuleMap.set(importStatement.moduleName, importStatement.subnode);
} }

23
server/src/analyzer/circularDependency.ts
View File

@ -1,14 +1,14 @@
/* /*
* circularDependency.ts * circularDependency.ts
* Copyright (c) Microsoft Corporation. * Copyright (c) Microsoft Corporation.
* Licensed under the MIT license. * Licensed under the MIT license.
* Author: Eric Traut * Author: Eric Traut
* *
* A list of file paths that are part of a circular dependency * A list of file paths that are part of a circular dependency
* chain (i.e. a chain of imports). Since these are circular, there * chain (i.e. a chain of imports). Since these are circular, there
* no defined "start", but this module helps normalize the start * no defined "start", but this module helps normalize the start
* by picking the alphabetically-first module in the cycle. * by picking the alphabetically-first module in the cycle.
*/ */
export class CircularDependency { export class CircularDependency {
private _paths: string[] = []; private _paths: string[] = [];
@ -32,8 +32,7 @@ export class CircularDependency {
}); });
if (firstIndex !== 0) { if (firstIndex !== 0) {
this._paths = this._paths.slice(firstIndex).concat( this._paths = this._paths.slice(firstIndex).concat(this._paths.slice(0, firstIndex));
this._paths.slice(0, firstIndex));
} }
} }

66
server/src/analyzer/codeFlow.ts
View File

@ -1,36 +1,42 @@
/* /*
* codeFlow.ts * codeFlow.ts
* Copyright (c) Microsoft Corporation. * Copyright (c) Microsoft Corporation.
* Licensed under the MIT license. * Licensed under the MIT license.
* Author: Eric Traut * Author: Eric Traut
* *
* Data structures that track the code flow (or more accurately, * Data structures that track the code flow (or more accurately,
* the inverse of code flow) starting with return statements and * the inverse of code flow) starting with return statements and
* working back to the entry. This allows us to work out the * working back to the entry. This allows us to work out the
* types at each point of the code flow. * types at each point of the code flow.
* *
* This is largely based on the code flow engine in the * This is largely based on the code flow engine in the
* TypeScript compiler. * TypeScript compiler.
*/ */
import { assert } from '../common/debug'; import { assert } from '../common/debug';
import { CallNode, ExpressionNode, ImportFromNode, MemberAccessNode, NameNode, import {
ParseNodeType } from '../parser/parseNodes'; CallNode,
ExpressionNode,
ImportFromNode,
MemberAccessNode,
NameNode,
ParseNodeType
} from '../parser/parseNodes';
export enum FlowFlags { export enum FlowFlags {
Unreachable = 1 << 0, // Unreachable code Unreachable = 1 << 0, // Unreachable code
Start = 1 << 1, // Entry point Start = 1 << 1, // Entry point
BranchLabel = 1 << 2, // Junction for forward control flow BranchLabel = 1 << 2, // Junction for forward control flow
LoopLabel = 1 << 3, // Junction for backward control flow LoopLabel = 1 << 3, // Junction for backward control flow
Assignment = 1 << 4, // Assignment statement Assignment = 1 << 4, // Assignment statement
Unbind = 1 << 5, // Used with assignment to indicate target should be unbound Unbind = 1 << 5, // Used with assignment to indicate target should be unbound
WildcardImport = 1 << 6, // For "from X import *" statements WildcardImport = 1 << 6, // For "from X import *" statements
TrueCondition = 1 << 7, // Condition known to be true TrueCondition = 1 << 7, // Condition known to be true
FalseCondition = 1 << 9, // Condition known to be false FalseCondition = 1 << 9, // Condition known to be false
Call = 1 << 10, // Call node Call = 1 << 10, // Call node
PreFinallyGate = 1 << 11, // Injected edge that links pre-finally label and pre-try flow PreFinallyGate = 1 << 11, // Injected edge that links pre-finally label and pre-try flow
PostFinally = 1 << 12, // Injected edge that links post-finally flow with the rest of the graph PostFinally = 1 << 12, // Injected edge that links post-finally flow with the rest of the graph
AssignmentAlias = 1 << 13 // Assigned symbol is aliased to another symbol with the same name AssignmentAlias = 1 << 13 // Assigned symbol is aliased to another symbol with the same name
} }
let _nextFlowNodeId = 1; let _nextFlowNodeId = 1;
@ -123,11 +129,11 @@ export function createKeyForReference(reference: NameNode | MemberAccessNode): s
key = reference.memberName.value; key = reference.memberName.value;
let leftNode = reference.leftExpression; let leftNode = reference.leftExpression;
while (leftNode.nodeType === ParseNodeType.MemberAccess) { while (leftNode.nodeType === ParseNodeType.MemberAccess) {
key = leftNode.memberName.value + `.${ key }`; key = leftNode.memberName.value + `.${key}`;
leftNode = leftNode.leftExpression; leftNode = leftNode.leftExpression;
} }
assert(leftNode.nodeType === ParseNodeType.Name); assert(leftNode.nodeType === ParseNodeType.Name);
key = (leftNode as NameNode).value + `.${ key }`; key = (leftNode as NameNode).value + `.${key}`;
} }
return key; return key;

34
server/src/analyzer/commentUtils.ts
View File

@ -1,21 +1,29 @@
/* /*
* commentUtils.ts * commentUtils.ts
* Copyright (c) Microsoft Corporation. * Copyright (c) Microsoft Corporation.
* Licensed under the MIT license. * Licensed under the MIT license.
* Author: Eric Traut * Author: Eric Traut
* *
* Utility functions that parse comments and extract commands * Utility functions that parse comments and extract commands
* or other directives from them. * or other directives from them.
*/ */
import { cloneDiagnosticSettings, DiagnosticLevel, DiagnosticSettings, import {
getBooleanDiagnosticSettings, getDiagLevelSettings, getStrictDiagnosticSettings } from '../common/configOptions'; cloneDiagnosticSettings,
DiagnosticLevel,
DiagnosticSettings,
getBooleanDiagnosticSettings,
getDiagLevelSettings,
getStrictDiagnosticSettings
} from '../common/configOptions';
import { TextRangeCollection } from '../common/textRangeCollection'; import { TextRangeCollection } from '../common/textRangeCollection';
import { Token } from '../parser/tokenizerTypes'; import { Token } from '../parser/tokenizerTypes';
export function getFileLevelDirectives(tokens: TextRangeCollection<Token>, export function getFileLevelDirectives(
defaultSettings: DiagnosticSettings, useStrict: boolean): DiagnosticSettings { tokens: TextRangeCollection<Token>,
defaultSettings: DiagnosticSettings,
useStrict: boolean
): DiagnosticSettings {
let settings = cloneDiagnosticSettings(defaultSettings); let settings = cloneDiagnosticSettings(defaultSettings);
if (useStrict) { if (useStrict) {

50
server/src/analyzer/declaration.ts
View File

@ -1,19 +1,32 @@
/* /*
* declaration.ts * declaration.ts
* Copyright (c) Microsoft Corporation. * Copyright (c) Microsoft Corporation.
* Licensed under the MIT license. * Licensed under the MIT license.
* Author: Eric Traut * Author: Eric Traut
* *
* Tracks the location within the code where a named entity * Tracks the location within the code where a named entity
* is declared and its associated declared type (if the type * is declared and its associated declared type (if the type
* is explicitly declared). * is explicitly declared).
*/ */
import { Range } from '../common/textRange'; import { Range } from '../common/textRange';
import { ClassNode, ExpressionNode, FunctionNode, ImportAsNode, import {
ImportFromAsNode, ImportFromNode, ModuleNode, NameNode, ParameterNode, ClassNode,
ParseNode, ReturnNode, StringListNode, TypeAnnotationNode, YieldFromNode, ExpressionNode,
YieldNode } from '../parser/parseNodes'; FunctionNode,
ImportAsNode,
ImportFromAsNode,
ImportFromNode,
ModuleNode,
NameNode,
ParameterNode,
ParseNode,
ReturnNode,
StringListNode,
TypeAnnotationNode,
YieldFromNode,
YieldNode
} from '../parser/parseNodes';
export const enum DeclarationType { export const enum DeclarationType {
Intrinsic, Intrinsic,
@ -132,6 +145,11 @@ export interface ModuleLoaderActions {
implicitImports?: Map<string, ModuleLoaderActions>; implicitImports?: Map<string, ModuleLoaderActions>;
} }
export type Declaration = IntrinsicDeclaration | ClassDeclaration | export type Declaration =
SpecialBuiltInClassDeclaration | FunctionDeclaration | ParameterDeclaration | | IntrinsicDeclaration
VariableDeclaration | AliasDeclaration; | ClassDeclaration
| SpecialBuiltInClassDeclaration
| FunctionDeclaration
| ParameterDeclaration
| VariableDeclaration
| AliasDeclaration;

22
server/src/analyzer/declarationUtils.ts
View File

@ -1,11 +1,11 @@
/* /*
* declarationUtils.ts * declarationUtils.ts
* Copyright (c) Microsoft Corporation. * Copyright (c) Microsoft Corporation.
* Licensed under the MIT license. * Licensed under the MIT license.
* Author: Eric Traut * Author: Eric Traut
* *
* Collection of static methods that operate on declarations. * Collection of static methods that operate on declarations.
*/ */
import { Declaration, DeclarationType } from './declaration'; import { Declaration, DeclarationType } from './declaration';
@ -37,8 +37,10 @@ export function areDeclarationsSame(decl1: Declaration, decl2: Declaration): boo
return false; return false;
} }
if (decl1.range.start.line !== decl2.range.start.line || if (
decl1.range.start.character !== decl2.range.start.character) { decl1.range.start.line !== decl2.range.start.line ||
decl1.range.start.character !== decl2.range.start.character
) {
return false; return false;
} }
@ -47,4 +49,4 @@ export function areDeclarationsSame(decl1: Declaration, decl2: Declaration): boo
export function isFinalVariableDeclaration(decl: Declaration) { export function isFinalVariableDeclaration(decl: Declaration) {
return decl.type === DeclarationType.Variable && !!decl.isFinal; return decl.type === DeclarationType.Variable && !!decl.isFinal;
} }

18
server/src/analyzer/docStringUtils.ts
View File

@ -1,13 +1,13 @@
/* /*
* docStringUtils.ts * docStringUtils.ts
* Copyright (c) Microsoft Corporation. * Copyright (c) Microsoft Corporation.
* Licensed under the MIT license. * Licensed under the MIT license.
* Author: Eric Traut * Author: Eric Traut
* *
* Static methods that format and parse doc strings based on * Static methods that format and parse doc strings based on
* the rules specified in PEP 257 * the rules specified in PEP 257
* (https://www.python.org/dev/peps/pep-0257/). * (https://www.python.org/dev/peps/pep-0257/).
*/ */
export function decodeDocString(rawString: string): string { export function decodeDocString(rawString: string): string {
// Remove carriage returns and replace tabs. // Remove carriage returns and replace tabs.

418
server/src/analyzer/importResolver.ts
View File

@ -1,18 +1,25 @@
/* /*
* importResolver.ts * importResolver.ts
* Copyright (c) Microsoft Corporation. * Copyright (c) Microsoft Corporation.
* Licensed under the MIT license. * Licensed under the MIT license.
* Author: Eric Traut * Author: Eric Traut
* *
* Provides the logic for resolving imports according to the * Provides the logic for resolving imports according to the
* runtime rules of Python. * runtime rules of Python.
*/ */
import { ConfigOptions, ExecutionEnvironment } from '../common/configOptions'; import { ConfigOptions, ExecutionEnvironment } from '../common/configOptions';
import { import {
combinePaths, ensureTrailingDirectorySeparator, getDirectoryPath, combinePaths,
getFileExtension, getFileSystemEntries, getPathComponents, isDirectory, ensureTrailingDirectorySeparator,
isFile, stripFileExtension, stripTrailingDirectorySeparator getDirectoryPath,
getFileExtension,
getFileSystemEntries,
getPathComponents,
isDirectory,
isFile,
stripFileExtension,
stripTrailingDirectorySeparator
} from '../common/pathUtils'; } from '../common/pathUtils';
import { versionToString } from '../common/pythonVersion'; import { versionToString } from '../common/pythonVersion';
import * as StringUtils from '../common/stringUtils'; import * as StringUtils from '../common/stringUtils';
@ -56,16 +63,22 @@ export class ImportResolver {
// Resolves the import and returns the path if it exists, otherwise // Resolves the import and returns the path if it exists, otherwise
// returns undefined. // returns undefined.
resolveImport(sourceFilePath: string, execEnv: ExecutionEnvironment, resolveImport(
moduleDescriptor: ImportedModuleDescriptor): ImportResult { sourceFilePath: string,
execEnv: ExecutionEnvironment,
moduleDescriptor: ImportedModuleDescriptor
): ImportResult {
const importName = this._formatImportName(moduleDescriptor); const importName = this._formatImportName(moduleDescriptor);
const importFailureInfo: string[] = []; const importFailureInfo: string[] = [];
// Is it a relative import? // Is it a relative import?
if (moduleDescriptor.leadingDots > 0) { if (moduleDescriptor.leadingDots > 0) {
const relativeImport = this._resolveRelativeImport(sourceFilePath, const relativeImport = this._resolveRelativeImport(
moduleDescriptor, importName, importFailureInfo); sourceFilePath,
moduleDescriptor,
importName,
importFailureInfo
);
if (relativeImport) { if (relativeImport) {
relativeImport.isRelative = true; relativeImport.isRelative = true;
@ -73,73 +86,91 @@ export class ImportResolver {
} }
} else { } else {
// Is it already cached? // Is it already cached?
const cachedResults = this._lookUpResultsInCache(execEnv, importName, const cachedResults = this._lookUpResultsInCache(execEnv, importName, moduleDescriptor.importedSymbols);
moduleDescriptor.importedSymbols);
if (cachedResults) { if (cachedResults) {
return cachedResults; return cachedResults;
} }
// First check for a typeshed file. // First check for a typeshed file.
if (moduleDescriptor.nameParts.length > 0) { if (moduleDescriptor.nameParts.length > 0) {
const builtInImport = this._findTypeshedPath(execEnv, moduleDescriptor, const builtInImport = this._findTypeshedPath(
importName, true, importFailureInfo); execEnv,
moduleDescriptor,
importName,
true,
importFailureInfo
);
if (builtInImport) { if (builtInImport) {
builtInImport.isTypeshedFile = true; builtInImport.isTypeshedFile = true;
return this._addResultsToCache(execEnv, importName, builtInImport, return this._addResultsToCache(
moduleDescriptor.importedSymbols); execEnv,
importName,
builtInImport,
moduleDescriptor.importedSymbols
);
} }
} }
let bestResultSoFar: ImportResult | undefined; let bestResultSoFar: ImportResult | undefined;
// Look for it in the root directory of the execution environment. // Look for it in the root directory of the execution environment.
importFailureInfo.push(`Looking in root directory of execution environment ` + importFailureInfo.push(`Looking in root directory of execution environment ` + `'${execEnv.root}'`);
`'${ execEnv.root }'`); let localImport = this.resolveAbsoluteImport(execEnv.root, moduleDescriptor, importName, importFailureInfo);
let localImport = this.resolveAbsoluteImport(
execEnv.root, moduleDescriptor, importName, importFailureInfo);
if (localImport && localImport.isImportFound) { if (localImport && localImport.isImportFound) {
return this._addResultsToCache(execEnv, importName, localImport, return this._addResultsToCache(execEnv, importName, localImport, moduleDescriptor.importedSymbols);
moduleDescriptor.importedSymbols);
} }
bestResultSoFar = localImport; bestResultSoFar = localImport;
for (const extraPath of execEnv.extraPaths) { for (const extraPath of execEnv.extraPaths) {
importFailureInfo.push(`Looking in extraPath '${ extraPath }'`); importFailureInfo.push(`Looking in extraPath '${extraPath}'`);
localImport = this.resolveAbsoluteImport(extraPath, moduleDescriptor, localImport = this.resolveAbsoluteImport(extraPath, moduleDescriptor, importName, importFailureInfo);
importName, importFailureInfo);
if (localImport && localImport.isImportFound) { if (localImport && localImport.isImportFound) {
return this._addResultsToCache(execEnv, importName, localImport, return this._addResultsToCache(execEnv, importName, localImport, moduleDescriptor.importedSymbols);
moduleDescriptor.importedSymbols);
} }
if (localImport && (bestResultSoFar === undefined || if (
localImport.resolvedPaths.length > bestResultSoFar.resolvedPaths.length)) { localImport &&
(bestResultSoFar === undefined ||
localImport.resolvedPaths.length > bestResultSoFar.resolvedPaths.length)
) {
bestResultSoFar = localImport; bestResultSoFar = localImport;
} }
} }
// Check for a typings file. // Check for a typings file.
if (this._configOptions.typingsPath) { if (this._configOptions.typingsPath) {
importFailureInfo.push(`Looking in typingsPath '${ this._configOptions.typingsPath }'`); importFailureInfo.push(`Looking in typingsPath '${this._configOptions.typingsPath}'`);
const typingsImport = this.resolveAbsoluteImport( const typingsImport = this.resolveAbsoluteImport(
this._configOptions.typingsPath, moduleDescriptor, importName, importFailureInfo); this._configOptions.typingsPath,
moduleDescriptor,
importName,
importFailureInfo
);
if (typingsImport && typingsImport.isImportFound) { if (typingsImport && typingsImport.isImportFound) {
// We will treat typings files as "local" rather than "third party". // We will treat typings files as "local" rather than "third party".
typingsImport.importType = ImportType.Local; typingsImport.importType = ImportType.Local;
typingsImport.isLocalTypingsFile = true; typingsImport.isLocalTypingsFile = true;
return this._addResultsToCache(execEnv, importName, typingsImport, return this._addResultsToCache(
moduleDescriptor.importedSymbols); execEnv,
importName,
typingsImport,
moduleDescriptor.importedSymbols
);
} }
} }
// Check for a typeshed file. // Check for a typeshed file.
importFailureInfo.push(`Looking for typeshed path`); importFailureInfo.push(`Looking for typeshed path`);
const typeshedImport = this._findTypeshedPath(execEnv, moduleDescriptor, const typeshedImport = this._findTypeshedPath(
importName, false, importFailureInfo); execEnv,
moduleDescriptor,
importName,
false,
importFailureInfo
);
if (typeshedImport) { if (typeshedImport) {
typeshedImport.isTypeshedFile = true; typeshedImport.isTypeshedFile = true;
return this._addResultsToCache(execEnv, importName, typeshedImport, return this._addResultsToCache(execEnv, importName, typeshedImport, moduleDescriptor.importedSymbols);
moduleDescriptor.importedSymbols);
} }
// Look for the import in the list of third-party packages. // Look for the import in the list of third-party packages.
@ -148,23 +179,35 @@ export class ImportResolver {
for (const searchPath of pythonSearchPaths) { for (const searchPath of pythonSearchPaths) {
// Allow partial resolution because some third-party packages // Allow partial resolution because some third-party packages
// use tricks to populate their package namespaces. // use tricks to populate their package namespaces.
importFailureInfo.push(`Looking in python search path '${ searchPath }'`); importFailureInfo.push(`Looking in python search path '${searchPath}'`);
const thirdPartyImport = this.resolveAbsoluteImport( const thirdPartyImport = this.resolveAbsoluteImport(
searchPath, moduleDescriptor, importName, importFailureInfo, searchPath,
true, true, true); moduleDescriptor,
importName,
importFailureInfo,
true,
true,
true
);
if (thirdPartyImport) { if (thirdPartyImport) {
thirdPartyImport.importType = ImportType.ThirdParty; thirdPartyImport.importType = ImportType.ThirdParty;
if (thirdPartyImport.isImportFound && thirdPartyImport.isStubFile) { if (thirdPartyImport.isImportFound && thirdPartyImport.isStubFile) {
return this._addResultsToCache(execEnv, importName, return this._addResultsToCache(
thirdPartyImport, moduleDescriptor.importedSymbols); execEnv,
importName,
thirdPartyImport,
moduleDescriptor.importedSymbols
);
} }
// We did not find it, or we did and it's not from a // We did not find it, or we did and it's not from a
// stub, so give chance for resolveImportEx to find // stub, so give chance for resolveImportEx to find
// one from a stub. // one from a stub.
if (bestResultSoFar === undefined || if (
thirdPartyImport.resolvedPaths.length > bestResultSoFar.resolvedPaths.length) { bestResultSoFar === undefined ||
thirdPartyImport.resolvedPaths.length > bestResultSoFar.resolvedPaths.length
) {
bestResultSoFar = thirdPartyImport; bestResultSoFar = thirdPartyImport;
} }
} }
@ -173,17 +216,21 @@ export class ImportResolver {
importFailureInfo.push('No python interpreter search path'); importFailureInfo.push('No python interpreter search path');
} }
const extraResults = this.resolveImportEx(sourceFilePath, execEnv, moduleDescriptor, importName, importFailureInfo); const extraResults = this.resolveImportEx(
sourceFilePath,
execEnv,
moduleDescriptor,
importName,
importFailureInfo
);
if (extraResults !== undefined) { if (extraResults !== undefined) {
return this._addResultsToCache(execEnv, importName, extraResults, return this._addResultsToCache(execEnv, importName, extraResults, moduleDescriptor.importedSymbols);
moduleDescriptor.importedSymbols);
} }
// We weren't able to find an exact match, so return the best // We weren't able to find an exact match, so return the best
// partial match. // partial match.
if (bestResultSoFar) { if (bestResultSoFar) {
return this._addResultsToCache(execEnv, importName, bestResultSoFar, return this._addResultsToCache(execEnv, importName, bestResultSoFar, moduleDescriptor.importedSymbols);
moduleDescriptor.importedSymbols);
} }
} }
@ -205,53 +252,64 @@ export class ImportResolver {
// Intended to be overridden by subclasses to provide additional stub // Intended to be overridden by subclasses to provide additional stub
// resolving capabilities. Return undefined if no stubs were found for // resolving capabilities. Return undefined if no stubs were found for
// this import. // this import.
protected resolveImportEx(sourceFilePath: string, execEnv: ExecutionEnvironment, protected resolveImportEx(
moduleDescriptor: ImportedModuleDescriptor, importName: string, sourceFilePath: string,
importFailureInfo: string[] = []): ImportResult | undefined { execEnv: ExecutionEnvironment,
moduleDescriptor: ImportedModuleDescriptor,
importName: string,
importFailureInfo: string[] = []
): ImportResult | undefined {
return undefined; return undefined;
} }
getCompletionSuggestions(sourceFilePath: string, execEnv: ExecutionEnvironment, getCompletionSuggestions(
moduleDescriptor: ImportedModuleDescriptor, similarityLimit: number): string[] { sourceFilePath: string,
execEnv: ExecutionEnvironment,
moduleDescriptor: ImportedModuleDescriptor,
similarityLimit: number
): string[] {
const importFailureInfo: string[] = []; const importFailureInfo: string[] = [];
const suggestions: string[] = []; const suggestions: string[] = [];
// Is it a relative import? // Is it a relative import?
if (moduleDescriptor.leadingDots > 0) { if (moduleDescriptor.leadingDots > 0) {
this._getCompletionSuggestionsRelative(sourceFilePath, this._getCompletionSuggestionsRelative(sourceFilePath, moduleDescriptor, suggestions, similarityLimit);
moduleDescriptor, suggestions, similarityLimit);
} else { } else {
// First check for a typeshed file. // First check for a typeshed file.
if (moduleDescriptor.nameParts.length > 0) { if (moduleDescriptor.nameParts.length > 0) {
this._getCompletionSuggestionsTypeshedPath(execEnv, moduleDescriptor, this._getCompletionSuggestionsTypeshedPath(
true, suggestions, similarityLimit); execEnv,
moduleDescriptor,
true,
suggestions,
similarityLimit
);
} }
// Look for it in the root directory of the execution environment. // Look for it in the root directory of the execution environment.
this._getCompletionSuggestionsAbsolute(execEnv.root, this._getCompletionSuggestionsAbsolute(execEnv.root, moduleDescriptor, suggestions, similarityLimit);
moduleDescriptor, suggestions, similarityLimit);
for (const extraPath of execEnv.extraPaths) { for (const extraPath of execEnv.extraPaths) {
this._getCompletionSuggestionsAbsolute(extraPath, moduleDescriptor, this._getCompletionSuggestionsAbsolute(extraPath, moduleDescriptor, suggestions, similarityLimit);
suggestions, similarityLimit);
} }
// Check for a typings file. // Check for a typings file.
if (this._configOptions.typingsPath) { if (this._configOptions.typingsPath) {
this._getCompletionSuggestionsAbsolute(this._configOptions.typingsPath, this._getCompletionSuggestionsAbsolute(
moduleDescriptor, suggestions, similarityLimit); this._configOptions.typingsPath,
moduleDescriptor,
suggestions,
similarityLimit
);
} }
// Check for a typeshed file. // Check for a typeshed file.
this._getCompletionSuggestionsTypeshedPath(execEnv, moduleDescriptor, this._getCompletionSuggestionsTypeshedPath(execEnv, moduleDescriptor, false, suggestions, similarityLimit);
false, suggestions, similarityLimit);
// Look for the import in the list of third-party packages. // Look for the import in the list of third-party packages.
const pythonSearchPaths = this._getPythonSearchPaths(execEnv, importFailureInfo); const pythonSearchPaths = this._getPythonSearchPaths(execEnv, importFailureInfo);
for (const searchPath of pythonSearchPaths) { for (const searchPath of pythonSearchPaths) {
this._getCompletionSuggestionsAbsolute(searchPath, this._getCompletionSuggestionsAbsolute(searchPath, moduleDescriptor, suggestions, similarityLimit);
moduleDescriptor, suggestions, similarityLimit);
} }
} }
@ -292,8 +350,7 @@ export class ImportResolver {
// Check for a typings file. // Check for a typings file.
if (this._configOptions.typingsPath) { if (this._configOptions.typingsPath) {
const candidateModuleName = this._getModuleNameFromPath( const candidateModuleName = this._getModuleNameFromPath(this._configOptions.typingsPath, filePath);
this._configOptions.typingsPath, filePath);
// Does this candidate look better than the previous best module name? // Does this candidate look better than the previous best module name?
// We'll always try to use the shortest version. // We'll always try to use the shortest version.
@ -337,9 +394,11 @@ export class ImportResolver {
return { moduleName: '', importType: ImportType.Local }; return { moduleName: '', importType: ImportType.Local };
} }
private _lookUpResultsInCache(execEnv: ExecutionEnvironment, importName: string, private _lookUpResultsInCache(
importedSymbols: string[] | undefined) { execEnv: ExecutionEnvironment,
importName: string,
importedSymbols: string[] | undefined
) {
const cacheForExecEnv = this._cachedImportResults.get(execEnv.root); const cacheForExecEnv = this._cachedImportResults.get(execEnv.root);
if (!cacheForExecEnv) { if (!cacheForExecEnv) {
return undefined; return undefined;
@ -353,9 +412,12 @@ export class ImportResolver {
return this._filterImplicitImports(cachedEntry, importedSymbols); return this._filterImplicitImports(cachedEntry, importedSymbols);
} }
private _addResultsToCache(execEnv: ExecutionEnvironment, importName: string, private _addResultsToCache(
importResult: ImportResult, importedSymbols: string[] | undefined) { execEnv: ExecutionEnvironment,
importName: string,
importResult: ImportResult,
importedSymbols: string[] | undefined
) {
let cacheForExecEnv = this._cachedImportResults.get(execEnv.root); let cacheForExecEnv = this._cachedImportResults.get(execEnv.root);
if (!cacheForExecEnv) { if (!cacheForExecEnv) {
cacheForExecEnv = new Map<string, ImportResult>(); cacheForExecEnv = new Map<string, ImportResult>();
@ -367,9 +429,11 @@ export class ImportResolver {
return this._filterImplicitImports(importResult, importedSymbols); return this._filterImplicitImports(importResult, importedSymbols);
} }
private _getModuleNameFromPath(containerPath: string, filePath: string, private _getModuleNameFromPath(
stripTopContainerDir = false): string | undefined { containerPath: string,
filePath: string,
stripTopContainerDir = false
): string | undefined {
containerPath = ensureTrailingDirectorySeparator(containerPath); containerPath = ensureTrailingDirectorySeparator(containerPath);
let filePathWithoutExtension = stripFileExtension(filePath); let filePathWithoutExtension = stripFileExtension(filePath);
@ -399,24 +463,33 @@ export class ImportResolver {
return parts.join('.'); return parts.join('.');
} }
private _getPythonSearchPaths(execEnv: ExecutionEnvironment, private _getPythonSearchPaths(execEnv: ExecutionEnvironment, importFailureInfo: string[]) {
importFailureInfo: string[]) {
const cacheKey = execEnv.venv ? execEnv.venv : '<default>'; const cacheKey = execEnv.venv ? execEnv.venv : '<default>';
// Find the site packages for the configured virtual environment. // Find the site packages for the configured virtual environment.
if (!this._cachedPythonSearchPaths.has(cacheKey)) { if (!this._cachedPythonSearchPaths.has(cacheKey)) {
this._cachedPythonSearchPaths.set(cacheKey, PythonPathUtils.findPythonSearchPaths( this._cachedPythonSearchPaths.set(
this.fileSystem, this._configOptions, execEnv.venv, importFailureInfo) || []); cacheKey,
PythonPathUtils.findPythonSearchPaths(
this.fileSystem,
this._configOptions,
execEnv.venv,
importFailureInfo
) || []
);
} }
return this._cachedPythonSearchPaths.get(cacheKey)!; return this._cachedPythonSearchPaths.get(cacheKey)!;
} }
private _findTypeshedPath(execEnv: ExecutionEnvironment, moduleDescriptor: ImportedModuleDescriptor, private _findTypeshedPath(
importName: string, isStdLib: boolean, importFailureInfo: string[]): ImportResult | undefined { execEnv: ExecutionEnvironment,
moduleDescriptor: ImportedModuleDescriptor,
importFailureInfo.push(`Looking for typeshed ${ isStdLib ? 'stdlib' : 'third_party' } path`); importName: string,
isStdLib: boolean,
importFailureInfo: string[]
): ImportResult | undefined {
importFailureInfo.push(`Looking for typeshed ${isStdLib ? 'stdlib' : 'third_party'} path`);
const typeshedPath = this._getTypeshedPath(isStdLib, execEnv, importFailureInfo); const typeshedPath = this._getTypeshedPath(isStdLib, execEnv, importFailureInfo);
if (!typeshedPath) { if (!typeshedPath) {
@ -424,16 +497,20 @@ export class ImportResolver {
} }
const pythonVersion = execEnv.pythonVersion; const pythonVersion = execEnv.pythonVersion;
let minorVersion = pythonVersion & 0xFF; let minorVersion = pythonVersion & 0xff;
// Search for module starting at "3.x" down to "3.1", then "3", then "2and3". // Search for module starting at "3.x" down to "3.1", then "3", then "2and3".
while (true) { while (true) {
const pythonVersionString = minorVersion > 0 ? versionToString(0x300 + minorVersion) : const pythonVersionString =
minorVersion === 0 ? '3' : '2and3'; minorVersion > 0 ? versionToString(0x300 + minorVersion) : minorVersion === 0 ? '3' : '2and3';
const testPath = combinePaths(typeshedPath, pythonVersionString); const testPath = combinePaths(typeshedPath, pythonVersionString);
if (this.fileSystem.existsSync(testPath)) { if (this.fileSystem.existsSync(testPath)) {
const importInfo = this.resolveAbsoluteImport(testPath, moduleDescriptor, const importInfo = this.resolveAbsoluteImport(
importName, importFailureInfo); testPath,
moduleDescriptor,
importName,
importFailureInfo
);
if (importInfo && importInfo.isImportFound) { if (importInfo && importInfo.isImportFound) {
importInfo.importType = isStdLib ? ImportType.BuiltIn : ImportType.ThirdParty; importInfo.importType = isStdLib ? ImportType.BuiltIn : ImportType.ThirdParty;
return importInfo; return importInfo;
@ -451,10 +528,13 @@ export class ImportResolver {
return undefined; return undefined;
} }
private _getCompletionSuggestionsTypeshedPath(execEnv: ExecutionEnvironment, private _getCompletionSuggestionsTypeshedPath(
moduleDescriptor: ImportedModuleDescriptor, isStdLib: boolean, execEnv: ExecutionEnvironment,
suggestions: string[], similarityLimit: number) { moduleDescriptor: ImportedModuleDescriptor,
isStdLib: boolean,
suggestions: string[],
similarityLimit: number
) {
const importFailureInfo: string[] = []; const importFailureInfo: string[] = [];
const typeshedPath = this._getTypeshedPath(isStdLib, execEnv, importFailureInfo); const typeshedPath = this._getTypeshedPath(isStdLib, execEnv, importFailureInfo);
if (!typeshedPath) { if (!typeshedPath) {
@ -462,16 +542,15 @@ export class ImportResolver {
} }
const pythonVersion = execEnv.pythonVersion; const pythonVersion = execEnv.pythonVersion;
let minorVersion = pythonVersion & 0xFF; let minorVersion = pythonVersion & 0xff;
// Search for module starting at "3.x" down to "3.1", then "3", then "2and3". // Search for module starting at "3.x" down to "3.1", then "3", then "2and3".
while (true) { while (true) {
const pythonVersionString = minorVersion > 0 ? versionToString(0x300 + minorVersion) : const pythonVersionString =
minorVersion === 0 ? '3' : '2and3'; minorVersion > 0 ? versionToString(0x300 + minorVersion) : minorVersion === 0 ? '3' : '2and3';
const testPath = combinePaths(typeshedPath, pythonVersionString); const testPath = combinePaths(typeshedPath, pythonVersionString);
if (this.fileSystem.existsSync(testPath)) { if (this.fileSystem.existsSync(testPath)) {
this._getCompletionSuggestionsAbsolute(testPath, moduleDescriptor, this._getCompletionSuggestionsAbsolute(testPath, moduleDescriptor, suggestions, similarityLimit);
suggestions, similarityLimit);
} }
// We use -1 to indicate "2and3", which is searched after "3.0". // We use -1 to indicate "2and3", which is searched after "3.0".
@ -482,9 +561,7 @@ export class ImportResolver {
} }
} }
private _getTypeshedPath(isStdLib: boolean, execEnv: ExecutionEnvironment, private _getTypeshedPath(isStdLib: boolean, execEnv: ExecutionEnvironment, importFailureInfo: string[]) {
importFailureInfo: string[]) {
// See if we have it cached. // See if we have it cached.
if (isStdLib) { if (isStdLib) {
if (this._cachedTypeshedStdLibPath !== undefined) { if (this._cachedTypeshedStdLibPath !== undefined) {
@ -502,14 +579,20 @@ export class ImportResolver {
// python search paths, then in the typeshed-fallback directory. // python search paths, then in the typeshed-fallback directory.
if (this._configOptions.typeshedPath) { if (this._configOptions.typeshedPath) {
const possibleTypeshedPath = this._configOptions.typeshedPath; const possibleTypeshedPath = this._configOptions.typeshedPath;
if (this.fileSystem.existsSync(possibleTypeshedPath) && isDirectory(this.fileSystem, possibleTypeshedPath)) { if (
this.fileSystem.existsSync(possibleTypeshedPath) &&
isDirectory(this.fileSystem, possibleTypeshedPath)
) {
typeshedPath = possibleTypeshedPath; typeshedPath = possibleTypeshedPath;
} }
} else { } else {
const pythonSearchPaths = this._getPythonSearchPaths(execEnv, importFailureInfo); const pythonSearchPaths = this._getPythonSearchPaths(execEnv, importFailureInfo);
for (const searchPath of pythonSearchPaths) { for (const searchPath of pythonSearchPaths) {
const possibleTypeshedPath = combinePaths(searchPath, 'typeshed'); const possibleTypeshedPath = combinePaths(searchPath, 'typeshed');
if (this.fileSystem.existsSync(possibleTypeshedPath) && isDirectory(this.fileSystem, possibleTypeshedPath)) { if (
this.fileSystem.existsSync(possibleTypeshedPath) &&
isDirectory(this.fileSystem, possibleTypeshedPath)
) {
typeshedPath = possibleTypeshedPath; typeshedPath = possibleTypeshedPath;
break; break;
} }
@ -537,25 +620,26 @@ export class ImportResolver {
return typeshedPath; return typeshedPath;
} }
private _resolveRelativeImport(sourceFilePath: string, private _resolveRelativeImport(
moduleDescriptor: ImportedModuleDescriptor, importName: string, sourceFilePath: string,
importFailureInfo: string[]): ImportResult | undefined { moduleDescriptor: ImportedModuleDescriptor,
importName: string,
importFailureInfo: string[]
): ImportResult | undefined {
importFailureInfo.push('Attempting to resolve relative import'); importFailureInfo.push('Attempting to resolve relative import');
// Determine which search path this file is part of. // Determine which search path this file is part of.
let curDir = getDirectoryPath(sourceFilePath); let curDir = getDirectoryPath(sourceFilePath);
for (let i = 1; i < moduleDescriptor.leadingDots; i++) { for (let i = 1; i < moduleDescriptor.leadingDots; i++) {
if (curDir === '') { if (curDir === '') {
importFailureInfo.push(`Invalid relative path '${ importName }'`); importFailureInfo.push(`Invalid relative path '${importName}'`);
return undefined; return undefined;
} }
curDir = getDirectoryPath(curDir); curDir = getDirectoryPath(curDir);
} }
// Now try to match the module parts from the current directory location. // Now try to match the module parts from the current directory location.
const absImport = this.resolveAbsoluteImport(curDir, moduleDescriptor, const absImport = this.resolveAbsoluteImport(curDir, moduleDescriptor, importName, importFailureInfo);
importName, importFailureInfo);
if (!absImport) { if (!absImport) {
return undefined; return undefined;
} }
@ -563,10 +647,12 @@ export class ImportResolver {
return this._filterImplicitImports(absImport, moduleDescriptor.importedSymbols); return this._filterImplicitImports(absImport, moduleDescriptor.importedSymbols);
} }
private _getCompletionSuggestionsRelative(sourceFilePath: string, private _getCompletionSuggestionsRelative(
moduleDescriptor: ImportedModuleDescriptor, suggestions: string[], sourceFilePath: string,
similarityLimit: number) { moduleDescriptor: ImportedModuleDescriptor,
suggestions: string[],
similarityLimit: number
) {
// Determine which search path this file is part of. // Determine which search path this file is part of.
let curDir = getDirectoryPath(sourceFilePath); let curDir = getDirectoryPath(sourceFilePath);
for (let i = 1; i < moduleDescriptor.leadingDots; i++) { for (let i = 1; i < moduleDescriptor.leadingDots; i++) {
@ -577,17 +663,21 @@ export class ImportResolver {
} }
// Now try to match the module parts from the current directory location. // Now try to match the module parts from the current directory location.
this._getCompletionSuggestionsAbsolute(curDir, moduleDescriptor, this._getCompletionSuggestionsAbsolute(curDir, moduleDescriptor, suggestions, similarityLimit);
suggestions, similarityLimit);
} }
// Follows import resolution algorithm defined in PEP-420: // Follows import resolution algorithm defined in PEP-420:
// https://www.python.org/dev/peps/pep-0420/ // https://www.python.org/dev/peps/pep-0420/
protected resolveAbsoluteImport(rootPath: string, moduleDescriptor: ImportedModuleDescriptor, protected resolveAbsoluteImport(
importName: string, importFailureInfo: string[], allowPartial = false, rootPath: string,
allowPydFile = false, allowStubsFolder = false): ImportResult | undefined { moduleDescriptor: ImportedModuleDescriptor,
importName: string,
importFailureInfo.push(`Attempting to resolve using root path '${ rootPath }'`); importFailureInfo: string[],
allowPartial = false,
allowPydFile = false,
allowStubsFolder = false
): ImportResult | undefined {
importFailureInfo.push(`Attempting to resolve using root path '${rootPath}'`);
// Starting at the specified path, walk the file system to find the // Starting at the specified path, walk the file system to find the
// specified module. // specified module.
@ -605,18 +695,22 @@ export class ImportResolver {
const pydFilePath = pyFilePath + 'd'; const pydFilePath = pyFilePath + 'd';
if (this.fileSystem.existsSync(pyiFilePath) && isFile(this.fileSystem, pyiFilePath)) { if (this.fileSystem.existsSync(pyiFilePath) && isFile(this.fileSystem, pyiFilePath)) {
importFailureInfo.push(`Resolved import with file '${ pyiFilePath }'`); importFailureInfo.push(`Resolved import with file '${pyiFilePath}'`);
resolvedPaths.push(pyiFilePath); resolvedPaths.push(pyiFilePath);
isStubFile = true; isStubFile = true;
} else if (this.fileSystem.existsSync(pyFilePath) && isFile(this.fileSystem, pyFilePath)) { } else if (this.fileSystem.existsSync(pyFilePath) && isFile(this.fileSystem, pyFilePath)) {
importFailureInfo.push(`Resolved import with file '${ pyFilePath }'`); importFailureInfo.push(`Resolved import with file '${pyFilePath}'`);
resolvedPaths.push(pyFilePath); resolvedPaths.push(pyFilePath);
} else if (allowPydFile && this.fileSystem.existsSync(pydFilePath) && isFile(this.fileSystem, pydFilePath)) { } else if (
importFailureInfo.push(`Resolved import with file '${ pydFilePath }'`); allowPydFile &&
this.fileSystem.existsSync(pydFilePath) &&
isFile(this.fileSystem, pydFilePath)
) {
importFailureInfo.push(`Resolved import with file '${pydFilePath}'`);
resolvedPaths.push(pydFilePath); resolvedPaths.push(pydFilePath);
isPydFile = true; isPydFile = true;
} else { } else {
importFailureInfo.push(`Partially resolved import with directory '${ dirPath }'`); importFailureInfo.push(`Partially resolved import with directory '${dirPath}'`);
resolvedPaths.push(''); resolvedPaths.push('');
isNamespacePackage = true; isNamespacePackage = true;
} }
@ -634,7 +728,8 @@ export class ImportResolver {
// the string '-stubs' to its top-level directory name. We'll // the string '-stubs' to its top-level directory name. We'll
// look there first. // look there first.
const stubsDirPath = dirPath + '-stubs'; const stubsDirPath = dirPath + '-stubs';
foundDirectory = this.fileSystem.existsSync(stubsDirPath) && isDirectory(this.fileSystem, stubsDirPath); foundDirectory =
this.fileSystem.existsSync(stubsDirPath) && isDirectory(this.fileSystem, stubsDirPath);
if (foundDirectory) { if (foundDirectory) {
dirPath = stubsDirPath; dirPath = stubsDirPath;
} }
@ -658,14 +753,14 @@ export class ImportResolver {
let foundInit = false; let foundInit = false;
if (this.fileSystem.existsSync(pyiFilePath) && isFile(this.fileSystem, pyiFilePath)) { if (this.fileSystem.existsSync(pyiFilePath) && isFile(this.fileSystem, pyiFilePath)) {
importFailureInfo.push(`Resolved import with file '${ pyiFilePath }'`); importFailureInfo.push(`Resolved import with file '${pyiFilePath}'`);
resolvedPaths.push(pyiFilePath); resolvedPaths.push(pyiFilePath);
if (isLastPart) { if (isLastPart) {
isStubFile = true; isStubFile = true;
} }
foundInit = true; foundInit = true;
} else if (this.fileSystem.existsSync(pyFilePath) && isFile(this.fileSystem, pyFilePath)) { } else if (this.fileSystem.existsSync(pyFilePath) && isFile(this.fileSystem, pyFilePath)) {
importFailureInfo.push(`Resolved import with file '${ pyFilePath }'`); importFailureInfo.push(`Resolved import with file '${pyFilePath}'`);
resolvedPaths.push(pyFilePath); resolvedPaths.push(pyFilePath);
foundInit = true; foundInit = true;
} }
@ -684,30 +779,35 @@ export class ImportResolver {
const pydFilePath = pyFilePath + 'd'; const pydFilePath = pyFilePath + 'd';
if (this.fileSystem.existsSync(pyiFilePath) && isFile(this.fileSystem, pyiFilePath)) { if (this.fileSystem.existsSync(pyiFilePath) && isFile(this.fileSystem, pyiFilePath)) {
importFailureInfo.push(`Resolved import with file '${ pyiFilePath }'`); importFailureInfo.push(`Resolved import with file '${pyiFilePath}'`);
resolvedPaths.push(pyiFilePath); resolvedPaths.push(pyiFilePath);
if (isLastPart) { if (isLastPart) {
isStubFile = true; isStubFile = true;
} }
} else if (this.fileSystem.existsSync(pyFilePath) && isFile(this.fileSystem, pyFilePath)) { } else if (this.fileSystem.existsSync(pyFilePath) && isFile(this.fileSystem, pyFilePath)) {
importFailureInfo.push(`Resolved import with file '${ pyFilePath }'`); importFailureInfo.push(`Resolved import with file '${pyFilePath}'`);
resolvedPaths.push(pyFilePath); resolvedPaths.push(pyFilePath);
} else if (allowPydFile && this.fileSystem.existsSync(pydFilePath) && isFile(this.fileSystem, pydFilePath)) { } else if (
importFailureInfo.push(`Resolved import with file '${ pydFilePath }'`); allowPydFile &&
this.fileSystem.existsSync(pydFilePath) &&
isFile(this.fileSystem, pydFilePath)
) {
importFailureInfo.push(`Resolved import with file '${pydFilePath}'`);
resolvedPaths.push(pydFilePath); resolvedPaths.push(pydFilePath);
if (isLastPart) { if (isLastPart) {
isPydFile = true; isPydFile = true;
} }
} else if (foundDirectory) { } else if (foundDirectory) {
importFailureInfo.push(`Partially resolved import with directory '${ dirPath }'`); importFailureInfo.push(`Partially resolved import with directory '${dirPath}'`);
resolvedPaths.push(''); resolvedPaths.push('');
if (isLastPart) { if (isLastPart) {
implicitImports = this._findImplicitImports(dirPath, [pyFilePath, pyiFilePath]); implicitImports = this._findImplicitImports(dirPath, [pyFilePath, pyiFilePath]);
isNamespacePackage = true; isNamespacePackage = true;
} }
} else { } else {
importFailureInfo.push(`Did not find file '${ pyiFilePath }',` + importFailureInfo.push(
` '${ pyFilePath }' or '${ pydFilePath }'`); `Did not find file '${pyiFilePath}',` + ` '${pyFilePath}' or '${pydFilePath}'`
);
} }
break; break;
} }
@ -739,10 +839,12 @@ export class ImportResolver {
}; };
} }
private _getCompletionSuggestionsAbsolute(rootPath: string, private _getCompletionSuggestionsAbsolute(
moduleDescriptor: ImportedModuleDescriptor, suggestions: string[], rootPath: string,
similarityLimit: number) { moduleDescriptor: ImportedModuleDescriptor,
suggestions: string[],
similarityLimit: number
) {
// Starting at the specified path, walk the file system to find the // Starting at the specified path, walk the file system to find the
// specified module. // specified module.
let dirPath = rootPath; let dirPath = rootPath;
@ -763,8 +865,7 @@ export class ImportResolver {
// Provide completions only if we're on the last part // Provide completions only if we're on the last part
// of the name. // of the name.
if (i === nameParts.length - 1) { if (i === nameParts.length - 1) {
this._addFilteredSuggestions(dirPath, this._addFilteredSuggestions(dirPath, nameParts[i], suggestions, similarityLimit);
nameParts[i], suggestions, similarityLimit);
} }
dirPath = combinePaths(dirPath, nameParts[i]); dirPath = combinePaths(dirPath, nameParts[i]);
@ -775,9 +876,7 @@ export class ImportResolver {
} }
} }
private _addFilteredSuggestions(dirPath: string, filter: string, suggestions: string[], private _addFilteredSuggestions(dirPath: string, filter: string, suggestions: string[], similarityLimit: number) {
similarityLimit: number) {
const entries = getFileSystemEntries(this.fileSystem, dirPath); const entries = getFileSystemEntries(this.fileSystem, dirPath);
entries.files.forEach(file => { entries.files.forEach(file => {
@ -786,9 +885,10 @@ export class ImportResolver {
if (fileExtension === '.py' || fileExtension === '.pyi' || fileExtension === '.pyd') { if (fileExtension === '.py' || fileExtension === '.pyi' || fileExtension === '.pyd') {
if (fileWithoutExtension !== '__init__') { if (fileWithoutExtension !== '__init__') {
if (!filter || StringUtils.computeCompletionSimilarity( if (
filter, fileWithoutExtension) >= similarityLimit) { !filter ||
StringUtils.computeCompletionSimilarity(filter, fileWithoutExtension) >= similarityLimit
) {
this._addUniqueSuggestion(fileWithoutExtension, suggestions); this._addUniqueSuggestion(fileWithoutExtension, suggestions);
} }
} }

14
server/src/analyzer/importResult.ts
View File

@ -1,11 +1,11 @@
/* /*
* importResult.ts * importResult.ts
* Copyright (c) Microsoft Corporation. * Copyright (c) Microsoft Corporation.
* Licensed under the MIT license. * Licensed under the MIT license.
* Author: Eric Traut * Author: Eric Traut
* *
* Interface that describes the output of the import resolver. * Interface that describes the output of the import resolver.
*/ */
export const enum ImportType { export const enum ImportType {
BuiltIn, BuiltIn,

96
server/src/analyzer/importStatementUtils.ts
View File

@ -1,19 +1,26 @@
/* /*
* importStatementUtils.ts * importStatementUtils.ts
* Copyright (c) Microsoft Corporation. * Copyright (c) Microsoft Corporation.
* Licensed under the MIT license. * Licensed under the MIT license.
* Author: Eric Traut * Author: Eric Traut
* *
* Utility routines for summarizing and manipulating * Utility routines for summarizing and manipulating
* import statements in a python source file. * import statements in a python source file.
*/ */
import { TextEditAction } from '../common/editAction'; import { TextEditAction } from '../common/editAction';
import { convertOffsetToPosition } from '../common/positionUtils'; import { convertOffsetToPosition } from '../common/positionUtils';
import { Position } from '../common/textRange'; import { Position } from '../common/textRange';
import { TextRange } from '../common/textRange'; import { TextRange } from '../common/textRange';
import { ImportAsNode, ImportFromAsNode, ImportFromNode, ImportNode, import {
ModuleNameNode, ModuleNode, ParseNodeType } from '../parser/parseNodes'; ImportAsNode,
ImportFromAsNode,
ImportFromNode,
ImportNode,
ModuleNameNode,
ModuleNode,
ParseNodeType
} from '../parser/parseNodes';
import { ParseResults } from '../parser/parser'; import { ParseResults } from '../parser/parser';
import * as AnalyzerNodeInfo from './analyzerNodeInfo'; import * as AnalyzerNodeInfo from './analyzerNodeInfo';
import { ImportResult, ImportType } from './importResult'; import { ImportResult, ImportType } from './importResult';
@ -67,9 +74,11 @@ export function getTopLevelImports(parseTree: ModuleNode): ImportStatements {
return localImports; return localImports;
} }
export function getTextEditsForAutoImportSymbolAddition(symbolName: string, export function getTextEditsForAutoImportSymbolAddition(
importStatement: ImportStatement, parseResults: ParseResults) { symbolName: string,
importStatement: ImportStatement,
parseResults: ParseResults
) {
const textEditList: TextEditAction[] = []; const textEditList: TextEditAction[] = [];
// Scan through the import symbols to find the right insertion point, // Scan through the import symbols to find the right insertion point,
@ -88,15 +97,16 @@ export function getTextEditsForAutoImportSymbolAddition(symbolName: string,
priorImport = curImport; priorImport = curImport;
} }
const insertionOffset = priorImport ? TextRange.getEnd(priorImport) : const insertionOffset = priorImport
(importStatement.node.imports.length > 0 ? ? TextRange.getEnd(priorImport)
importStatement.node.imports[0].start : : importStatement.node.imports.length > 0
importStatement.node.start + importStatement.node.length); ? importStatement.node.imports[0].start
: importStatement.node.start + importStatement.node.length;
const insertionPosition = convertOffsetToPosition(insertionOffset, parseResults.tokenizerOutput.lines); const insertionPosition = convertOffsetToPosition(insertionOffset, parseResults.tokenizerOutput.lines);
textEditList.push({ textEditList.push({
range: { start: insertionPosition, end: insertionPosition }, range: { start: insertionPosition, end: insertionPosition },
replacementText: priorImport ? (', ' + symbolName) : (symbolName + ', ') replacementText: priorImport ? ', ' + symbolName : symbolName + ', '
}); });
} }
} }
@ -104,13 +114,17 @@ export function getTextEditsForAutoImportSymbolAddition(symbolName: string,
return textEditList; return textEditList;
} }
export function getTextEditsForAutoImportInsertion(symbolName: string, importStatements: ImportStatements, export function getTextEditsForAutoImportInsertion(
moduleName: string, importType: ImportType, parseResults: ParseResults): TextEditAction[] { symbolName: string,
importStatements: ImportStatements,
moduleName: string,
importType: ImportType,
parseResults: ParseResults
): TextEditAction[] {
const textEditList: TextEditAction[] = []; const textEditList: TextEditAction[] = [];
// We need to emit a new 'from import' statement. // We need to emit a new 'from import' statement.
let newImportStatement = `from ${ moduleName } import ${ symbolName }`; let newImportStatement = `from ${moduleName} import ${symbolName}`;
let insertionPosition: Position; let insertionPosition: Position;
if (importStatements.orderedImports.length > 0) { if (importStatements.orderedImports.length > 0) {
let insertBefore = true; let insertBefore = true;
@ -124,8 +138,9 @@ export function getTextEditsForAutoImportInsertion(symbolName: string, importSta
// If the import was resolved, use its import type. If it wasn't // If the import was resolved, use its import type. If it wasn't
// resolved, assume that it's the same import type as the previous // resolved, assume that it's the same import type as the previous
// one. // one.
const curImportType: ImportType = curImport.importResult ? const curImportType: ImportType = curImport.importResult
curImport.importResult.importType : prevImportType; ? curImport.importResult.importType
: prevImportType;
if (importType < curImportType) { if (importType < curImportType) {
if (!insertBefore && prevImportType < importType) { if (!insertBefore && prevImportType < importType) {
@ -151,7 +166,6 @@ export function getTextEditsForAutoImportInsertion(symbolName: string, importSta
// If this is the last import, see if we need to create a new group. // If this is the last import, see if we need to create a new group.
if (curImport === importStatements.orderedImports[importStatements.orderedImports.length - 1]) { if (curImport === importStatements.orderedImports[importStatements.orderedImports.length - 1]) {
if (importType > curImportType) { if (importType > curImportType) {
// Add an extra line to create a new group. // Add an extra line to create a new group.
newImportStatement = parseResults.tokenizerOutput.predominantEndOfLineSequence + newImportStatement; newImportStatement = parseResults.tokenizerOutput.predominantEndOfLineSequence + newImportStatement;
@ -178,7 +192,8 @@ export function getTextEditsForAutoImportInsertion(symbolName: string, importSta
insertionPosition = convertOffsetToPosition( insertionPosition = convertOffsetToPosition(
insertBefore ? insertionImport.node.start : TextRange.getEnd(insertionImport.node), insertBefore ? insertionImport.node.start : TextRange.getEnd(insertionImport.node),
parseResults.tokenizerOutput.lines); parseResults.tokenizerOutput.lines
);
} else { } else {
insertionPosition = { line: 0, character: 0 }; insertionPosition = { line: 0, character: 0 };
} }
@ -207,19 +222,20 @@ export function getTextEditsForAutoImportInsertion(symbolName: string, importSta
} }
if (stopHere) { if (stopHere) {
insertionPosition = convertOffsetToPosition(statement.start, insertionPosition = convertOffsetToPosition(statement.start, parseResults.tokenizerOutput.lines);
parseResults.tokenizerOutput.lines);
addNewLineBefore = false; addNewLineBefore = false;
break; break;
} else { } else {
insertionPosition = convertOffsetToPosition( insertionPosition = convertOffsetToPosition(
statement.start + statement.length, statement.start + statement.length,
parseResults.tokenizerOutput.lines); parseResults.tokenizerOutput.lines
);
addNewLineBefore = true; addNewLineBefore = true;
} }
} }
newImportStatement += parseResults.tokenizerOutput.predominantEndOfLineSequence + newImportStatement +=
parseResults.tokenizerOutput.predominantEndOfLineSequence +
parseResults.tokenizerOutput.predominantEndOfLineSequence; parseResults.tokenizerOutput.predominantEndOfLineSequence;
if (addNewLineBefore) { if (addNewLineBefore) {
@ -237,9 +253,7 @@ export function getTextEditsForAutoImportInsertion(symbolName: string, importSta
return textEditList; return textEditList;
} }
function _processImportNode(node: ImportNode, localImports: ImportStatements, function _processImportNode(node: ImportNode, localImports: ImportStatements, followsNonImportStatement: boolean) {
followsNonImportStatement: boolean) {
node.list.forEach(importAsNode => { node.list.forEach(importAsNode => {
const importResult = AnalyzerNodeInfo.getImportInfo(importAsNode.module); const importResult = AnalyzerNodeInfo.getImportInfo(importAsNode.module);
let resolvedPath: string | undefined; let resolvedPath: string | undefined;
@ -271,9 +285,11 @@ function _processImportNode(node: ImportNode, localImports: ImportStatements,
}); });
} }
function _processImportFromNode(node: ImportFromNode, localImports: ImportStatements, function _processImportFromNode(
followsNonImportStatement: boolean) { node: ImportFromNode,
localImports: ImportStatements,
followsNonImportStatement: boolean
) {
const importResult = AnalyzerNodeInfo.getImportInfo(node.module); const importResult = AnalyzerNodeInfo.getImportInfo(node.module);
let resolvedPath: string | undefined; let resolvedPath: string | undefined;
@ -297,9 +313,11 @@ function _processImportFromNode(node: ImportFromNode, localImports: ImportStatem
// Overwrite existing import statements because we always want to prefer // Overwrite existing import statements because we always want to prefer
// 'import from' over 'import'. Also, overwrite existing 'import from' if // 'import from' over 'import'. Also, overwrite existing 'import from' if
// the module name is shorter. // the module name is shorter.
if (!prevEntry || prevEntry.node.nodeType === ParseNodeType.Import || if (
prevEntry.moduleName.length > localImport.moduleName.length) { !prevEntry ||
prevEntry.node.nodeType === ParseNodeType.Import ||
prevEntry.moduleName.length > localImport.moduleName.length
) {
localImports.mapByFilePath.set(resolvedPath, localImport); localImports.mapByFilePath.set(resolvedPath, localImport);
} }
} }

24
server/src/analyzer/parseTreeCleaner.ts
View File

@ -1,15 +1,15 @@
/* /*
* parseTreeCleaner.ts * parseTreeCleaner.ts
* Copyright (c) Microsoft Corporation. * Copyright (c) Microsoft Corporation.
* Licensed under the MIT license. * Licensed under the MIT license.
* Author: Eric Traut * Author: Eric Traut
* *
* A parse tree walker that's used to clean any analysis * A parse tree walker that's used to clean any analysis
* information hanging off the parse tree. It's used when * information hanging off the parse tree. It's used when
* dependent files have been modified and the file requires * dependent files have been modified and the file requires
* reanalysis. Without this, we'd need to generate a fresh * reanalysis. Without this, we'd need to generate a fresh
* parse tree from scratch. * parse tree from scratch.
*/ */
import { ModuleNode, ParseNode } from '../parser/parseNodes'; import { ModuleNode, ParseNode } from '../parser/parseNodes';
import * as AnalyzerNodeInfo from './analyzerNodeInfo'; import * as AnalyzerNodeInfo from './analyzerNodeInfo';
@ -30,6 +30,6 @@ export class ParseTreeCleanerWalker extends ParseTreeWalker {
visitNode(node: ParseNode) { visitNode(node: ParseNode) {
AnalyzerNodeInfo.cleanNodeAnalysisInfo(node); AnalyzerNodeInfo.cleanNodeAnalysisInfo(node);
return super.visitNode(node); return super.visitNode(node);
} }
} }

315
server/src/analyzer/parseTreeUtils.ts
View File

@ -1,20 +1,34 @@
/* /*
* parseTreeUtils.ts * parseTreeUtils.ts
* Copyright (c) Microsoft Corporation. * Copyright (c) Microsoft Corporation.
* Licensed under the MIT license. * Licensed under the MIT license.
* Author: Eric Traut * Author: Eric Traut
* *
* Utility routines for traversing a parse tree. * Utility routines for traversing a parse tree.
*/ */
import { fail } from '../common/debug'; import { fail } from '../common/debug';
import { convertPositionToOffset } from '../common/positionUtils'; import { convertPositionToOffset } from '../common/positionUtils';
import { Position } from '../common/textRange'; import { Position } from '../common/textRange';
import { TextRange } from '../common/textRange'; import { TextRange } from '../common/textRange';
import { TextRangeCollection } from '../common/textRangeCollection'; import { TextRangeCollection } from '../common/textRangeCollection';
import { ArgumentCategory, AssignmentExpressionNode, ClassNode, EvaluationScopeNode, import {
ExecutionScopeNode, ExpressionNode, FunctionNode, isExpressionNode, LambdaNode, ModuleNode, ArgumentCategory,
ParameterCategory, ParseNode, ParseNodeType, StatementNode, SuiteNode } from '../parser/parseNodes'; AssignmentExpressionNode,
ClassNode,
EvaluationScopeNode,
ExecutionScopeNode,
ExpressionNode,
FunctionNode,
isExpressionNode,
LambdaNode,
ModuleNode,
ParameterCategory,
ParseNode,
ParseNodeType,
StatementNode,
SuiteNode
} from '../parser/parseNodes';
import { KeywordType, OperatorType, StringTokenFlags } from '../parser/tokenizerTypes'; import { KeywordType, OperatorType, StringTokenFlags } from '../parser/tokenizerTypes';
import { decodeDocString } from './docStringUtils'; import { decodeDocString } from './docStringUtils';
import { ParseTreeWalker } from './parseTreeWalker'; import { ParseTreeWalker } from './parseTreeWalker';
@ -39,9 +53,11 @@ export function getNodeDepth(node: ParseNode): number {
} }
// Returns the deepest node that contains the specified position. // Returns the deepest node that contains the specified position.
export function findNodeByPosition(node: ParseNode, position: Position, export function findNodeByPosition(
lines: TextRangeCollection<TextRange>): ParseNode | undefined { node: ParseNode,
position: Position,
lines: TextRangeCollection<TextRange>
): ParseNode | undefined {
const offset = convertPositionToOffset(position, lines); const offset = convertPositionToOffset(position, lines);
if (offset === undefined) { if (offset === undefined) {
return undefined; return undefined;
@ -80,43 +96,53 @@ export function printExpression(node: ExpressionNode, flags = PrintExpressionFla
} }
case ParseNodeType.MemberAccess: { case ParseNodeType.MemberAccess: {
return printExpression(node.leftExpression, flags) + '.' + return printExpression(node.leftExpression, flags) + '.' + node.memberName.value;
node.memberName.value;
} }
case ParseNodeType.Call: { case ParseNodeType.Call: {
return printExpression(node.leftExpression, flags) + '(' + return (
node.arguments.map(arg => { printExpression(node.leftExpression, flags) +
let argStr = ''; '(' +
if (arg.argumentCategory === ArgumentCategory.UnpackedList) { node.arguments
argStr = '*'; .map(arg => {
} else if (arg.argumentCategory === ArgumentCategory.UnpackedDictionary) { let argStr = '';
argStr = '**'; if (arg.argumentCategory === ArgumentCategory.UnpackedList) {
} argStr = '*';
if (arg.name) { } else if (arg.argumentCategory === ArgumentCategory.UnpackedDictionary) {
argStr += arg.name.value + '='; argStr = '**';
} }
argStr += printExpression(arg.valueExpression, flags); if (arg.name) {
return argStr; argStr += arg.name.value + '=';
}).join(', ') + }
')'; argStr += printExpression(arg.valueExpression, flags);
return argStr;
})
.join(', ') +
')'
);
} }
case ParseNodeType.Index: { case ParseNodeType.Index: {
return printExpression(node.baseExpression, flags) + '[' + return (
printExpression(node.baseExpression, flags) +
'[' +
node.items.items.map(item => printExpression(item, flags)).join(', ') + node.items.items.map(item => printExpression(item, flags)).join(', ') +
']'; ']'
);
} }
case ParseNodeType.UnaryOperation: { case ParseNodeType.UnaryOperation: {
return printOperator(node.operator) + ' ' + return printOperator(node.operator) + ' ' + printExpression(node.expression, flags);
printExpression(node.expression, flags);
} }
case ParseNodeType.BinaryOperation: { case ParseNodeType.BinaryOperation: {
return printExpression(node.leftExpression, flags) + ' ' + return (
printOperator(node.operator) + ' ' + printExpression(node.leftExpression, flags) +
printExpression(node.rightExpression, flags); ' ' +
printOperator(node.operator) +
' ' +
printExpression(node.rightExpression, flags)
);
} }
case ParseNodeType.Number: { case ParseNodeType.Number: {
@ -128,12 +154,14 @@ export function printExpression(node: ExpressionNode, flags = PrintExpressionFla
} }
case ParseNodeType.StringList: { case ParseNodeType.StringList: {
if ((flags & PrintExpressionFlags.ForwardDeclarations) && node.typeAnnotation) { if (flags & PrintExpressionFlags.ForwardDeclarations && node.typeAnnotation) {
return printExpression(node.typeAnnotation, flags); return printExpression(node.typeAnnotation, flags);
} else { } else {
return node.strings.map(str => { return node.strings
return printExpression(str, flags); .map(str => {
}).join(' '); return printExpression(str, flags);
})
.join(' ');
} }
} }
@ -157,15 +185,15 @@ export function printExpression(node: ExpressionNode, flags = PrintExpressionFla
if (node.token.flags & StringTokenFlags.Triplicate) { if (node.token.flags & StringTokenFlags.Triplicate) {
if (node.token.flags & StringTokenFlags.SingleQuote) { if (node.token.flags & StringTokenFlags.SingleQuote) {
exprString += `'''${ node.token.escapedValue }'''`; exprString += `'''${node.token.escapedValue}'''`;
} else { } else {
exprString += `"""${ node.token.escapedValue }"""`; exprString += `"""${node.token.escapedValue}"""`;
} }
} else { } else {
if (node.token.flags & StringTokenFlags.SingleQuote) { if (node.token.flags & StringTokenFlags.SingleQuote) {
exprString += `'${ node.token.escapedValue }'`; exprString += `'${node.token.escapedValue}'`;
} else { } else {
exprString += `"${ node.token.escapedValue }"`; exprString += `"${node.token.escapedValue}"`;
} }
} }
@ -173,24 +201,25 @@ export function printExpression(node: ExpressionNode, flags = PrintExpressionFla
} }
case ParseNodeType.Assignment: { case ParseNodeType.Assignment: {
return printExpression(node.leftExpression, flags) + ' = ' + return printExpression(node.leftExpression, flags) + ' = ' + printExpression(node.rightExpression, flags);
printExpression(node.rightExpression, flags);
} }
case ParseNodeType.AssignmentExpression: { case ParseNodeType.AssignmentExpression: {
return printExpression(node.name, flags) + ' := ' + return printExpression(node.name, flags) + ' := ' + printExpression(node.rightExpression, flags);
printExpression(node.rightExpression, flags);
} }
case ParseNodeType.TypeAnnotation: { case ParseNodeType.TypeAnnotation: {
return printExpression(node.valueExpression, flags) + ': ' + return printExpression(node.valueExpression, flags) + ': ' + printExpression(node.typeAnnotation, flags);
printExpression(node.typeAnnotation, flags);
} }
case ParseNodeType.AugmentedAssignment: { case ParseNodeType.AugmentedAssignment: {
return printExpression(node.leftExpression, flags) + ' ' + return (
printOperator(node.operator) + ' ' + printExpression(node.leftExpression, flags) +
printExpression(node.rightExpression, flags); ' ' +
printOperator(node.operator) +
' ' +
printExpression(node.rightExpression, flags)
);
} }
case ParseNodeType.Await: { case ParseNodeType.Await: {
@ -198,16 +227,20 @@ export function printExpression(node: ExpressionNode, flags = PrintExpressionFla
} }
case ParseNodeType.Ternary: { case ParseNodeType.Ternary: {
return printExpression(node.ifExpression, flags) + ' if ' + return (
printExpression(node.testExpression, flags) + ' else ' + printExpression(node.ifExpression, flags) +
printExpression(node.elseExpression, flags); ' if ' +
printExpression(node.testExpression, flags) +
' else ' +
printExpression(node.elseExpression, flags)
);
} }
case ParseNodeType.List: { case ParseNodeType.List: {
const expressions = node.entries.map(expr => { const expressions = node.entries.map(expr => {
return printExpression(expr, flags); return printExpression(expr, flags);
}); });
return `[${ expressions.join(', ') }]`; return `[${expressions.join(', ')}]`;
} }
case ParseNodeType.Unpack: { case ParseNodeType.Unpack: {
@ -219,9 +252,9 @@ export function printExpression(node: ExpressionNode, flags = PrintExpressionFla
return printExpression(expr, flags); return printExpression(expr, flags);
}); });
if (expressions.length === 1) { if (expressions.length === 1) {
return `(${ expressions[0] }, )`; return `(${expressions[0]}, )`;
} }
return `(${ expressions.join(', ') })`; return `(${expressions.join(', ')})`;
} }
case ParseNodeType.Yield: { case ParseNodeType.Yield: {
@ -248,19 +281,26 @@ export function printExpression(node: ExpressionNode, flags = PrintExpressionFla
} else if (node.expression.nodeType === ParseNodeType.DictionaryKeyEntry) { } else if (node.expression.nodeType === ParseNodeType.DictionaryKeyEntry) {
const keyStr = printExpression(node.expression.keyExpression, flags); const keyStr = printExpression(node.expression.keyExpression, flags);
const valueStr = printExpression(node.expression.valueExpression, flags); const valueStr = printExpression(node.expression.valueExpression, flags);
listStr = `${ keyStr }: ${ valueStr }`; listStr = `${keyStr}: ${valueStr}`;
} }
return listStr + ' ' + return (
node.comprehensions.map(expr => { listStr +
if (expr.nodeType === ParseNodeType.ListComprehensionFor) { ' ' +
return `${ expr.isAsync ? 'async ' : '' }for ` + node.comprehensions
printExpression(expr.targetExpression, flags) + .map(expr => {
` in ${ printExpression(expr.iterableExpression, flags) }`; if (expr.nodeType === ParseNodeType.ListComprehensionFor) {
} else { return (
return `if ${ printExpression(expr.testExpression, flags) }`; `${expr.isAsync ? 'async ' : ''}for ` +
} printExpression(expr.targetExpression, flags) +
}).join(' '); ` in ${printExpression(expr.iterableExpression, flags)}`
);
} else {
return `if ${printExpression(expr.testExpression, flags)}`;
}
})
.join(' ')
);
} }
case ParseNodeType.Slice: { case ParseNodeType.Slice: {
@ -278,24 +318,31 @@ export function printExpression(node: ExpressionNode, flags = PrintExpressionFla
} }
case ParseNodeType.Lambda: { case ParseNodeType.Lambda: {
return 'lambda ' + node.parameters.map(param => { return (
let paramStr = ''; 'lambda ' +
node.parameters
.map(param => {
let paramStr = '';
if (param.category === ParameterCategory.VarArgList) { if (param.category === ParameterCategory.VarArgList) {
paramStr += '*'; paramStr += '*';
} else if (param.category === ParameterCategory.VarArgDictionary) { } else if (param.category === ParameterCategory.VarArgDictionary) {
paramStr += '**'; paramStr += '**';
} }
if (param.name) { if (param.name) {
paramStr += param.name.value; paramStr += param.name.value;
} }
if (param.defaultValue) { if (param.defaultValue) {
paramStr += ' = ' + printExpression(param.defaultValue, flags); paramStr += ' = ' + printExpression(param.defaultValue, flags);
} }
return paramStr; return paramStr;
}).join(', ') + ': ' + printExpression(node.expression, flags); })
.join(', ') +
': ' +
printExpression(node.expression, flags)
);
} }
case ParseNodeType.Constant: { case ParseNodeType.Constant: {
@ -312,10 +359,12 @@ export function printExpression(node: ExpressionNode, flags = PrintExpressionFla
} }
case ParseNodeType.Dictionary: { case ParseNodeType.Dictionary: {
return `{ ${ node.entries.map(entry => { return `{ ${node.entries.map(entry => {
if (entry.nodeType === ParseNodeType.DictionaryKeyEntry) { if (entry.nodeType === ParseNodeType.DictionaryKeyEntry) {
return `${ printExpression(entry.keyExpression, flags) }: ` + return (
`${ printExpression(entry.valueExpression, flags) }`; `${printExpression(entry.keyExpression, flags)}: ` +
`${printExpression(entry.valueExpression, flags)}`
);
} else { } else {
return printExpression(entry, flags); return printExpression(entry, flags);
} }
@ -323,7 +372,7 @@ export function printExpression(node: ExpressionNode, flags = PrintExpressionFla
} }
case ParseNodeType.DictionaryExpandEntry: { case ParseNodeType.DictionaryExpandEntry: {
return `**${ printExpression(node.expandExpression, flags) }`; return `**${printExpression(node.expandExpression, flags)}`;
} }
case ParseNodeType.Set: { case ParseNodeType.Set: {
@ -423,9 +472,7 @@ export function getEnclosingModule(node: ParseNode): ModuleNode {
return undefined!; return undefined!;
} }
export function getEnclosingClassOrModule(node: ParseNode, export function getEnclosingClassOrModule(node: ParseNode, stopAtFunction = false): ClassNode | ModuleNode | undefined {
stopAtFunction = false): ClassNode | ModuleNode | undefined {
let curNode = node.parent; let curNode = node.parent;
while (curNode) { while (curNode) {
if (curNode.nodeType === ParseNodeType.Class) { if (curNode.nodeType === ParseNodeType.Class) {
@ -465,9 +512,9 @@ export function getEnclosingFunction(node: ParseNode): FunctionNode | undefined
return undefined; return undefined;
} }
export function getEvaluationNodeForAssignmentExpression(node: AssignmentExpressionNode): export function getEvaluationNodeForAssignmentExpression(
LambdaNode | FunctionNode | ModuleNode | undefined { node: AssignmentExpressionNode
): LambdaNode | FunctionNode | ModuleNode | undefined {
// PEP 572 indicates that the evaluation node for an assignment expression // PEP 572 indicates that the evaluation node for an assignment expression
// target is the containing lambda, function or module, but not a class. // target is the containing lambda, function or module, but not a class.
let curNode: ParseNode | undefined = getEvaluationScopeNode(node); let curNode: ParseNode | undefined = getEvaluationScopeNode(node);
@ -547,9 +594,10 @@ export function getExecutionScopeNode(node: ParseNode): ExecutionScopeNode {
// Classes are not considered execution scope because they are executed // Classes are not considered execution scope because they are executed
// within the context of their containing module or function. Likewise, list // within the context of their containing module or function. Likewise, list
// comprehensions are executed within their container. // comprehensions are executed within their container.
while (evaluationScope.nodeType === ParseNodeType.Class || while (
evaluationScope.nodeType === ParseNodeType.ListComprehension) { evaluationScope.nodeType === ParseNodeType.Class ||
evaluationScope.nodeType === ParseNodeType.ListComprehension
) {
evaluationScope = getEvaluationScopeNode(evaluationScope.parent!); evaluationScope = getEvaluationScopeNode(evaluationScope.parent!);
} }
@ -592,9 +640,14 @@ export function isSuiteEmpty(node: SuiteNode): boolean {
export function isMatchingExpression(expression1: ExpressionNode, expression2: ExpressionNode): boolean { export function isMatchingExpression(expression1: ExpressionNode, expression2: ExpressionNode): boolean {
if (expression1.nodeType === ParseNodeType.Name && expression2.nodeType === ParseNodeType.Name) { if (expression1.nodeType === ParseNodeType.Name && expression2.nodeType === ParseNodeType.Name) {
return expression1.value === expression2.value; return expression1.value === expression2.value;
} else if (expression1.nodeType === ParseNodeType.MemberAccess && expression2.nodeType === ParseNodeType.MemberAccess) { } else if (
return isMatchingExpression(expression1.leftExpression, expression2.leftExpression) && expression1.nodeType === ParseNodeType.MemberAccess &&
expression1.memberName.value === expression2.memberName.value; expression2.nodeType === ParseNodeType.MemberAccess
) {
return (
isMatchingExpression(expression1.leftExpression, expression2.leftExpression) &&
expression1.memberName.value === expression2.memberName.value
);
} }
return false; return false;
@ -609,10 +662,12 @@ export function isWithinDefaultParamInitializer(node: ParseNode) {
return true; return true;
} }
if (curNode.nodeType === ParseNodeType.Lambda || if (
curNode.nodeType === ParseNodeType.Function || curNode.nodeType === ParseNodeType.Lambda ||
curNode.nodeType === ParseNodeType.Class || curNode.nodeType === ParseNodeType.Function ||
curNode.nodeType === ParseNodeType.Module) { curNode.nodeType === ParseNodeType.Class ||
curNode.nodeType === ParseNodeType.Module
) {
return false; return false;
} }
@ -636,8 +691,7 @@ export function getDocString(statements: StatementNode[]): string | undefined {
// If the first statement in the suite isn't a StringNode, // If the first statement in the suite isn't a StringNode,
// assume there is no docString. // assume there is no docString.
const statementList = statements[0]; const statementList = statements[0];
if (statementList.statements.length === 0 || if (statementList.statements.length === 0 || statementList.statements[0].nodeType !== ParseNodeType.StringList) {
statementList.statements[0].nodeType !== ParseNodeType.StringList) {
return undefined; return undefined;
} }
@ -658,22 +712,27 @@ export function getDocString(statements: StatementNode[]): string | undefined {
// This pattern is commonly used to set the default values that are // This pattern is commonly used to set the default values that are
// not specified in the original list. // not specified in the original list.
export function isAssignmentToDefaultsFollowingNamedTuple(callNode: ParseNode): boolean { export function isAssignmentToDefaultsFollowingNamedTuple(callNode: ParseNode): boolean {
if (callNode.nodeType !== ParseNodeType.Call || !callNode.parent || if (
callNode.parent.nodeType !== ParseNodeType.Assignment || callNode.nodeType !== ParseNodeType.Call ||
callNode.parent.leftExpression.nodeType !== ParseNodeType.Name || !callNode.parent ||
!callNode.parent.parent || callNode.parent.nodeType !== ParseNodeType.Assignment ||
callNode.parent.parent.nodeType !== ParseNodeType.StatementList) { callNode.parent.leftExpression.nodeType !== ParseNodeType.Name ||
!callNode.parent.parent ||
callNode.parent.parent.nodeType !== ParseNodeType.StatementList
) {
return false; return false;
} }
const namedTupleAssignedName = callNode.parent.leftExpression.value; const namedTupleAssignedName = callNode.parent.leftExpression.value;
const statementList = callNode.parent.parent; const statementList = callNode.parent.parent;
if (statementList.statements[0] !== callNode.parent || if (
!statementList.parent || statementList.statements[0] !== callNode.parent ||
!(statementList.parent.nodeType === ParseNodeType.Module || !statementList.parent ||
statementList.parent.nodeType === ParseNodeType.Suite)) { !(
statementList.parent.nodeType === ParseNodeType.Module ||
statementList.parent.nodeType === ParseNodeType.Suite
)
) {
return false; return false;
} }
@ -699,15 +758,17 @@ export function isAssignmentToDefaultsFollowingNamedTuple(callNode: ParseNode):
if (nextStatement.statements[0].nodeType === ParseNodeType.Assignment) { if (nextStatement.statements[0].nodeType === ParseNodeType.Assignment) {
const assignNode = nextStatement.statements[0]; const assignNode = nextStatement.statements[0];
if (assignNode.leftExpression.nodeType === ParseNodeType.MemberAccess && if (
assignNode.leftExpression.memberName.value === '__defaults__') { assignNode.leftExpression.nodeType === ParseNodeType.MemberAccess &&
assignNode.leftExpression.memberName.value === '__defaults__'
) {
const defaultTarget = assignNode.leftExpression.leftExpression; const defaultTarget = assignNode.leftExpression.leftExpression;
if (defaultTarget.nodeType === ParseNodeType.MemberAccess && if (
defaultTarget.memberName.value === '__new__' && defaultTarget.nodeType === ParseNodeType.MemberAccess &&
defaultTarget.leftExpression.nodeType === ParseNodeType.Name && defaultTarget.memberName.value === '__new__' &&
defaultTarget.leftExpression.value === namedTupleAssignedName) { defaultTarget.leftExpression.nodeType === ParseNodeType.Name &&
defaultTarget.leftExpression.value === namedTupleAssignedName
) {
return true; return true;
} }
} }

98
server/src/analyzer/parseTreeWalker.ts
View File

@ -1,26 +1,81 @@
/* /*
* parseTreeWalker.ts * parseTreeWalker.ts
* Copyright (c) Microsoft Corporation. * Copyright (c) Microsoft Corporation.
* Licensed under the MIT license. * Licensed under the MIT license.
* Author: Eric Traut * Author: Eric Traut
* *
* Class that traverses a parse tree. * Class that traverses a parse tree.
*/ */
import { fail } from '../common/debug'; import { fail } from '../common/debug';
import { ArgumentNode, AssertNode, AssignmentExpressionNode, AssignmentNode, import {
AugmentedAssignmentNode, AwaitNode, BinaryOperationNode, BreakNode, CallNode, ArgumentNode,
ClassNode, ConstantNode, ContinueNode, DecoratorNode, DelNode, AssertNode,
DictionaryExpandEntryNode, DictionaryKeyEntryNode, DictionaryNode, EllipsisNode, AssignmentExpressionNode,
ErrorNode, ExceptNode, FormatStringNode, ForNode, FunctionNode, GlobalNode, IfNode, AssignmentNode,
ImportAsNode, ImportFromAsNode, ImportFromNode, ImportNode, IndexItemsNode, AugmentedAssignmentNode,
IndexNode, LambdaNode, ListComprehensionForNode, ListComprehensionIfNode, AwaitNode,
ListComprehensionNode, ListNode, MemberAccessNode, ModuleNameNode, ModuleNode, NameNode, BinaryOperationNode,
NonlocalNode, NumberNode, ParameterNode, ParseNode, ParseNodeArray, ParseNodeType, BreakNode,
PassNode, RaiseNode, ReturnNode, SetNode, SliceNode, StatementListNode, CallNode,
StringListNode, StringNode, SuiteNode, TernaryNode, TryNode, ClassNode,
TupleNode, TypeAnnotationNode, UnaryOperationNode, UnpackNode, ConstantNode,
WhileNode, WithItemNode, WithNode, YieldFromNode, YieldNode } from '../parser/parseNodes'; ContinueNode,
DecoratorNode,
DelNode,
DictionaryExpandEntryNode,
DictionaryKeyEntryNode,
DictionaryNode,
EllipsisNode,
ErrorNode,
ExceptNode,
FormatStringNode,
ForNode,
FunctionNode,
GlobalNode,
IfNode,
ImportAsNode,
ImportFromAsNode,
ImportFromNode,
ImportNode,
IndexItemsNode,
IndexNode,
LambdaNode,
ListComprehensionForNode,
ListComprehensionIfNode,
ListComprehensionNode,
ListNode,
MemberAccessNode,
ModuleNameNode,
ModuleNode,
NameNode,
NonlocalNode,
NumberNode,
ParameterNode,
ParseNode,
ParseNodeArray,
ParseNodeType,
PassNode,
RaiseNode,
ReturnNode,
SetNode,
SliceNode,
StatementListNode,
StringListNode,
StringNode,
SuiteNode,
TernaryNode,
TryNode,
TupleNode,
TypeAnnotationNode,
UnaryOperationNode,
UnpackNode,
WhileNode,
WithItemNode,
WithNode,
YieldFromNode,
YieldNode
} from '../parser/parseNodes';
// To use this class, create a subclass and override the // To use this class, create a subclass and override the
// visitXXX methods that you want to handle. // visitXXX methods that you want to handle.
@ -229,8 +284,7 @@ export class ParseTreeWalker {
case ParseNodeType.Function: case ParseNodeType.Function:
if (this.visitFunction(node)) { if (this.visitFunction(node)) {
return [...node.decorators, node.name, ...node.parameters, return [...node.decorators, node.name, ...node.parameters, node.returnTypeAnnotation, node.suite];
node.returnTypeAnnotation, node.suite];
} }
break; break;

265
server/src/analyzer/program.ts
View File

@ -1,12 +1,12 @@
/* /*
* program.ts * program.ts
* Copyright (c) Microsoft Corporation. * Copyright (c) Microsoft Corporation.
* Licensed under the MIT license. * Licensed under the MIT license.
* Author: Eric Traut * Author: Eric Traut
* *
* An object that tracks all of the source files being analyzed * An object that tracks all of the source files being analyzed
* and all of their recursive imports. * and all of their recursive imports.
*/ */
import { CompletionItem, CompletionList, DocumentSymbol, SymbolInformation } from 'vscode-languageserver'; import { CompletionItem, CompletionList, DocumentSymbol, SymbolInformation } from 'vscode-languageserver';
@ -17,8 +17,12 @@ import { Diagnostic } from '../common/diagnostic';
import { FileDiagnostics } from '../common/diagnosticSink'; import { FileDiagnostics } from '../common/diagnosticSink';
import { FileEditAction, TextEditAction } from '../common/editAction'; import { FileEditAction, TextEditAction } from '../common/editAction';
import { import {
combinePaths, getDirectoryPath, getRelativePath, makeDirectories, combinePaths,
normalizePath, stripFileExtension getDirectoryPath,
getRelativePath,
makeDirectories,
normalizePath,
stripFileExtension
} from '../common/pathUtils'; } from '../common/pathUtils';
import { DocumentRange, doRangesOverlap, Position, Range } from '../common/textRange'; import { DocumentRange, doRangesOverlap, Position, Range } from '../common/textRange';
import { Duration, timingStats } from '../common/timing'; import { Duration, timingStats } from '../common/timing';
@ -82,8 +86,11 @@ export class Program {
private _configOptions: ConfigOptions; private _configOptions: ConfigOptions;
private _importResolver: ImportResolver; private _importResolver: ImportResolver;
constructor(initialImportResolver: ImportResolver, initialConfigOptions: ConfigOptions, constructor(
console?: ConsoleInterface) { initialImportResolver: ImportResolver,
initialConfigOptions: ConfigOptions,
console?: ConsoleInterface
) {
this._console = console || new StandardConsole(); this._console = console || new StandardConsole();
this._evaluator = createTypeEvaluator(this._lookUpImport); this._evaluator = createTypeEvaluator(this._lookUpImport);
this._importResolver = initialImportResolver; this._importResolver = initialImportResolver;
@ -140,10 +147,11 @@ export class Program {
let sourceFileCount = 0; let sourceFileCount = 0;
this._sourceFileList.forEach(fileInfo => { this._sourceFileList.forEach(fileInfo => {
if (fileInfo.sourceFile.isParseRequired() || if (
fileInfo.sourceFile.isParseRequired() ||
fileInfo.sourceFile.isBindingRequired() || fileInfo.sourceFile.isBindingRequired() ||
fileInfo.sourceFile.isCheckingRequired()) { fileInfo.sourceFile.isCheckingRequired()
) {
if ((!this._configOptions.checkOnlyOpenFiles && fileInfo.isTracked) || fileInfo.isOpenByClient) { if ((!this._configOptions.checkOnlyOpenFiles && fileInfo.isTracked) || fileInfo.isOpenByClient) {
sourceFileCount++; sourceFileCount++;
} }
@ -272,13 +280,10 @@ export class Program {
analyze(maxTime?: MaxAnalysisTime, interactiveMode?: boolean): boolean { analyze(maxTime?: MaxAnalysisTime, interactiveMode?: boolean): boolean {
const elapsedTime = new Duration(); const elapsedTime = new Duration();
const openFiles = this._sourceFileList.filter( const openFiles = this._sourceFileList.filter(sf => sf.isOpenByClient && sf.sourceFile.isCheckingRequired());
sf => sf.isOpenByClient && sf.sourceFile.isCheckingRequired()
);
if (openFiles.length > 0) { if (openFiles.length > 0) {
const effectiveMaxTime = maxTime ? const effectiveMaxTime = maxTime ? maxTime.openFilesTimeInMs : Number.MAX_VALUE;
maxTime.openFilesTimeInMs : Number.MAX_VALUE;
// Check the open files. // Check the open files.
for (const sourceFileInfo of openFiles) { for (const sourceFileInfo of openFiles) {
@ -300,9 +305,11 @@ export class Program {
if (!this._configOptions.checkOnlyOpenFiles) { if (!this._configOptions.checkOnlyOpenFiles) {
// Do type analysis of remaining files. // Do type analysis of remaining files.
const allFiles = this._sourceFileList; const allFiles = this._sourceFileList;
const effectiveMaxTime = maxTime ? const effectiveMaxTime = maxTime
(interactiveMode ? maxTime.openFilesTimeInMs : maxTime.noOpenFilesTimeInMs) : ? interactiveMode
Number.MAX_VALUE; ? maxTime.openFilesTimeInMs
: maxTime.noOpenFilesTimeInMs
: Number.MAX_VALUE;
// Now do type parsing and analysis of the remaining. // Now do type parsing and analysis of the remaining.
for (const sourceFileInfo of allFiles) { for (const sourceFileInfo of allFiles) {
@ -320,9 +327,11 @@ export class Program {
// Prints import dependency information for each of the files in // Prints import dependency information for each of the files in
// the program, skipping any typeshed files. // the program, skipping any typeshed files.
printDependencies(projectRootDir: string, verbose: boolean) { printDependencies(projectRootDir: string, verbose: boolean) {
const sortedFiles = this._sourceFileList.filter(s => !s.isTypeshedFile).sort((a, b) => { const sortedFiles = this._sourceFileList
return (a.sourceFile.getFilePath() < b.sourceFile.getFilePath()) ? 1 : -1; .filter(s => !s.isTypeshedFile)
}); .sort((a, b) => {
return a.sourceFile.getFilePath() < b.sourceFile.getFilePath() ? 1 : -1;
});
const zeroImportFiles: SourceFile[] = []; const zeroImportFiles: SourceFile[] = [];
@ -334,21 +343,23 @@ export class Program {
filePath = relPath; filePath = relPath;
} }
this._console.log(`${ filePath }`); this._console.log(`${filePath}`);
this._console.log(` Imports ${ sfInfo.imports.length } ` + this._console.log(
`file${ sfInfo.imports.length === 1 ? '' : 's' }`); ` Imports ${sfInfo.imports.length} ` + `file${sfInfo.imports.length === 1 ? '' : 's'}`
);
if (verbose) { if (verbose) {
sfInfo.imports.forEach(importInfo => { sfInfo.imports.forEach(importInfo => {
this._console.log(` ${ importInfo.sourceFile.getFilePath() }`); this._console.log(` ${importInfo.sourceFile.getFilePath()}`);
}); });
} }
this._console.log(` Imported by ${ sfInfo.importedBy.length } ` + this._console.log(
`file${ sfInfo.importedBy.length === 1 ? '' : 's' }`); ` Imported by ${sfInfo.importedBy.length} ` + `file${sfInfo.importedBy.length === 1 ? '' : 's'}`
);
if (verbose) { if (verbose) {
sfInfo.importedBy.forEach(importInfo => { sfInfo.importedBy.forEach(importInfo => {
this._console.log(` ${ importInfo.sourceFile.getFilePath() }`); this._console.log(` ${importInfo.sourceFile.getFilePath()}`);
}); });
} }
@ -359,10 +370,11 @@ export class Program {
if (zeroImportFiles.length > 0) { if (zeroImportFiles.length > 0) {
this._console.log(''); this._console.log('');
this._console.log(`${ zeroImportFiles.length } file${ zeroImportFiles.length === 1 ? '' : 's' }` + this._console.log(
` not explicitly imported`); `${zeroImportFiles.length} file${zeroImportFiles.length === 1 ? '' : 's'}` + ` not explicitly imported`
);
zeroImportFiles.forEach(importFile => { zeroImportFiles.forEach(importFile => {
this._console.log(` ${ importFile.getFilePath() }`); this._console.log(` ${importFile.getFilePath()}`);
}); });
} }
} }
@ -391,7 +403,7 @@ export class Program {
try { try {
makeDirectories(this._fs, typeStubDir, typingsPath); makeDirectories(this._fs, typeStubDir, typingsPath);
} catch (e) { } catch (e) {
const errMsg = `Could not create directory for '${ typeStubDir }'`; const errMsg = `Could not create directory for '${typeStubDir}'`;
throw new Error(errMsg); throw new Error(errMsg);
} }
@ -482,7 +494,7 @@ export class Program {
symbolTable, symbolTable,
docString docString
}; };
} };
// Build a map of all modules within this program and the module- // Build a map of all modules within this program and the module-
// level scope that contains the symbol table for the module. // level scope that contains the symbol table for the module.
@ -553,9 +565,11 @@ export class Program {
// it imports (recursively) and ensures that all such files. If any of these files // it imports (recursively) and ensures that all such files. If any of these files
// have already been checked (they and their recursive imports have completed the // have already been checked (they and their recursive imports have completed the
// check phase), they are not included in the results. // check phase), they are not included in the results.
private _getImportsRecursive(file: SourceFileInfo, closureMap: Map<string, SourceFileInfo>, private _getImportsRecursive(
recursionCount: number) { file: SourceFileInfo,
closureMap: Map<string, SourceFileInfo>,
recursionCount: number
) {
// If the file is already in the closure map, we found a cyclical // If the file is already in the closure map, we found a cyclical
// dependency. Don't recur further. // dependency. Don't recur further.
const filePath = file.sourceFile.getFilePath(); const filePath = file.sourceFile.getFilePath();
@ -579,10 +593,11 @@ export class Program {
} }
} }
private _detectAndReportImportCycles(sourceFileInfo: SourceFileInfo, private _detectAndReportImportCycles(
sourceFileInfo: SourceFileInfo,
dependencyChain: SourceFileInfo[] = [], dependencyChain: SourceFileInfo[] = [],
dependencyMap = new Map<string, boolean>()): void { dependencyMap = new Map<string, boolean>()
): void {
// Don't bother checking for typestub files or third-party files. // Don't bother checking for typestub files or third-party files.
if (sourceFileInfo.sourceFile.isStubFile() || sourceFileInfo.isThirdPartyImport) { if (sourceFileInfo.sourceFile.isStubFile() || sourceFileInfo.isThirdPartyImport) {
return; return;
@ -634,9 +649,7 @@ export class Program {
firstSourceFile.sourceFile.addCircularDependency(circDep); firstSourceFile.sourceFile.addCircularDependency(circDep);
} }
private _markFileDirtyRecursive(sourceFileInfo: SourceFileInfo, private _markFileDirtyRecursive(sourceFileInfo: SourceFileInfo, markMap: Map<string, boolean>) {
markMap: Map<string, boolean>) {
const filePath = sourceFileInfo.sourceFile.getFilePath(); const filePath = sourceFileInfo.sourceFile.getFilePath();
// Don't mark it again if it's already been visited. // Don't mark it again if it's already been visited.
@ -656,7 +669,9 @@ export class Program {
this._sourceFileList.forEach(sourceFileInfo => { this._sourceFileList.forEach(sourceFileInfo => {
if ((!options.checkOnlyOpenFiles && sourceFileInfo.isTracked) || sourceFileInfo.isOpenByClient) { if ((!options.checkOnlyOpenFiles && sourceFileInfo.isTracked) || sourceFileInfo.isOpenByClient) {
const diagnostics = sourceFileInfo.sourceFile.getDiagnostics( const diagnostics = sourceFileInfo.sourceFile.getDiagnostics(
options, sourceFileInfo.diagnosticsVersion); options,
sourceFileInfo.diagnosticsVersion
);
if (diagnostics !== undefined) { if (diagnostics !== undefined) {
fileDiagnostics.push({ fileDiagnostics.push({
filePath: sourceFileInfo.sourceFile.getFilePath(), filePath: sourceFileInfo.sourceFile.getFilePath(),
@ -665,8 +680,7 @@ export class Program {
// Update the cached diagnosticsVersion so we can determine // Update the cached diagnosticsVersion so we can determine
// whether there are any updates next time we call getDiagnostics. // whether there are any updates next time we call getDiagnostics.
sourceFileInfo.diagnosticsVersion = sourceFileInfo.diagnosticsVersion = sourceFileInfo.sourceFile.getDiagnosticVersion();
sourceFileInfo.sourceFile.getDiagnosticVersion();
} }
} }
}); });
@ -690,9 +704,7 @@ export class Program {
}); });
} }
getDefinitionsForPosition(filePath: string, position: Position): getDefinitionsForPosition(filePath: string, position: Position): DocumentRange[] | undefined {
DocumentRange[] | undefined {
const sourceFileInfo = this._sourceFileMap.get(filePath); const sourceFileInfo = this._sourceFileMap.get(filePath);
if (!sourceFileInfo) { if (!sourceFileInfo) {
return undefined; return undefined;
@ -703,9 +715,11 @@ export class Program {
return sourceFileInfo.sourceFile.getDefinitionsForPosition(position, this._evaluator); return sourceFileInfo.sourceFile.getDefinitionsForPosition(position, this._evaluator);
} }
getReferencesForPosition(filePath: string, position: Position, getReferencesForPosition(
includeDeclaration: boolean): DocumentRange[] | undefined { filePath: string,
position: Position,
includeDeclaration: boolean
): DocumentRange[] | undefined {
const sourceFileInfo = this._sourceFileMap.get(filePath); const sourceFileInfo = this._sourceFileMap.get(filePath);
if (!sourceFileInfo) { if (!sourceFileInfo) {
return undefined; return undefined;
@ -714,7 +728,10 @@ export class Program {
this._bindFile(sourceFileInfo); this._bindFile(sourceFileInfo);
const referencesResult = sourceFileInfo.sourceFile.getReferencesForPosition( const referencesResult = sourceFileInfo.sourceFile.getReferencesForPosition(
position, includeDeclaration, this._evaluator); position,
includeDeclaration,
this._evaluator
);
if (!referencesResult) { if (!referencesResult) {
return undefined; return undefined;
@ -726,8 +743,7 @@ export class Program {
if (curSourceFileInfo !== sourceFileInfo) { if (curSourceFileInfo !== sourceFileInfo) {
this._bindFile(curSourceFileInfo); this._bindFile(curSourceFileInfo);
curSourceFileInfo.sourceFile.addReferences(referencesResult, curSourceFileInfo.sourceFile.addReferences(referencesResult, includeDeclaration, this._evaluator);
includeDeclaration, this._evaluator);
} }
} }
} }
@ -740,8 +756,7 @@ export class Program {
if (sourceFileInfo) { if (sourceFileInfo) {
this._bindFile(sourceFileInfo); this._bindFile(sourceFileInfo);
sourceFileInfo.sourceFile.addHierarchicalSymbolsForDocument( sourceFileInfo.sourceFile.addHierarchicalSymbolsForDocument(symbolList, this._evaluator);
symbolList, this._evaluator);
} }
} }
@ -755,14 +770,11 @@ export class Program {
for (const sourceFileInfo of this._sourceFileList) { for (const sourceFileInfo of this._sourceFileList) {
this._bindFile(sourceFileInfo); this._bindFile(sourceFileInfo);
sourceFileInfo.sourceFile.addSymbolsForDocument( sourceFileInfo.sourceFile.addSymbolsForDocument(symbolList, this._evaluator, query);
symbolList, this._evaluator, query);
} }
} }
getHoverForPosition(filePath: string, position: Position): getHoverForPosition(filePath: string, position: Position): HoverResults | undefined {
HoverResults | undefined {
const sourceFileInfo = this._sourceFileMap.get(filePath); const sourceFileInfo = this._sourceFileMap.get(filePath);
if (!sourceFileInfo) { if (!sourceFileInfo) {
return undefined; return undefined;
@ -773,9 +785,7 @@ export class Program {
return sourceFileInfo.sourceFile.getHoverForPosition(position, this._evaluator); return sourceFileInfo.sourceFile.getHoverForPosition(position, this._evaluator);
} }
getSignatureHelpForPosition(filePath: string, position: Position): getSignatureHelpForPosition(filePath: string, position: Position): SignatureHelpResults | undefined {
SignatureHelpResults | undefined {
const sourceFileInfo = this._sourceFileMap.get(filePath); const sourceFileInfo = this._sourceFileMap.get(filePath);
if (!sourceFileInfo) { if (!sourceFileInfo) {
return undefined; return undefined;
@ -783,13 +793,10 @@ export class Program {
this._bindFile(sourceFileInfo); this._bindFile(sourceFileInfo);
return sourceFileInfo.sourceFile.getSignatureHelpForPosition( return sourceFileInfo.sourceFile.getSignatureHelpForPosition(position, this._lookUpImport, this._evaluator);
position, this._lookUpImport, this._evaluator);
} }
getCompletionsForPosition(filePath: string, position: Position, getCompletionsForPosition(filePath: string, position: Position, workspacePath: string): CompletionList | undefined {
workspacePath: string): CompletionList | undefined {
const sourceFileInfo = this._sourceFileMap.get(filePath); const sourceFileInfo = this._sourceFileMap.get(filePath);
if (!sourceFileInfo) { if (!sourceFileInfo) {
return undefined; return undefined;
@ -798,9 +805,14 @@ export class Program {
this._bindFile(sourceFileInfo); this._bindFile(sourceFileInfo);
return sourceFileInfo.sourceFile.getCompletionsForPosition( return sourceFileInfo.sourceFile.getCompletionsForPosition(
position, workspacePath, this._configOptions, position,
this._importResolver, this._lookUpImport, this._evaluator, workspacePath,
() => this._buildModuleSymbolsMap(sourceFileInfo)); this._configOptions,
this._importResolver,
this._lookUpImport,
this._evaluator,
() => this._buildModuleSymbolsMap(sourceFileInfo)
);
} }
resolveCompletionItem(filePath: string, completionItem: CompletionItem) { resolveCompletionItem(filePath: string, completionItem: CompletionItem) {
@ -812,13 +824,16 @@ export class Program {
this._bindFile(sourceFileInfo); this._bindFile(sourceFileInfo);
sourceFileInfo.sourceFile.resolveCompletionItem( sourceFileInfo.sourceFile.resolveCompletionItem(
this._configOptions, this._importResolver, this._lookUpImport, this._evaluator, this._configOptions,
() => this._buildModuleSymbolsMap(sourceFileInfo), completionItem); this._importResolver,
this._lookUpImport,
this._evaluator,
() => this._buildModuleSymbolsMap(sourceFileInfo),
completionItem
);
} }
performQuickAction(filePath: string, command: string, performQuickAction(filePath: string, command: string, args: any[]): TextEditAction[] | undefined {
args: any[]): TextEditAction[] | undefined {
const sourceFileInfo = this._sourceFileMap.get(filePath); const sourceFileInfo = this._sourceFileMap.get(filePath);
if (!sourceFileInfo) { if (!sourceFileInfo) {
return undefined; return undefined;
@ -826,20 +841,16 @@ export class Program {
this._bindFile(sourceFileInfo); this._bindFile(sourceFileInfo);
return sourceFileInfo.sourceFile.performQuickAction( return sourceFileInfo.sourceFile.performQuickAction(command, args);
command, args);
} }
renameSymbolAtPosition(filePath: string, position: Position, renameSymbolAtPosition(filePath: string, position: Position, newName: string): FileEditAction[] | undefined {
newName: string): FileEditAction[] | undefined {
const sourceFileInfo = this._sourceFileMap.get(filePath); const sourceFileInfo = this._sourceFileMap.get(filePath);
if (!sourceFileInfo) { if (!sourceFileInfo) {
return undefined; return undefined;
} }
const referencesResult = sourceFileInfo.sourceFile.getReferencesForPosition( const referencesResult = sourceFileInfo.sourceFile.getReferencesForPosition(position, true, this._evaluator);
position, true, this._evaluator);
if (!referencesResult) { if (!referencesResult) {
return undefined; return undefined;
@ -851,8 +862,7 @@ export class Program {
if (curSourceFileInfo !== sourceFileInfo) { if (curSourceFileInfo !== sourceFileInfo) {
this._bindFile(curSourceFileInfo); this._bindFile(curSourceFileInfo);
curSourceFileInfo.sourceFile.addReferences(referencesResult, curSourceFileInfo.sourceFile.addReferences(referencesResult, true, this._evaluator);
true, this._evaluator);
} }
} }
} }
@ -878,7 +888,7 @@ export class Program {
// If a file is no longer tracked or opened, it can // If a file is no longer tracked or opened, it can
// be removed from the program. // be removed from the program.
for (let i = 0; i < this._sourceFileList.length;) { for (let i = 0; i < this._sourceFileList.length; ) {
const fileInfo = this._sourceFileList[i]; const fileInfo = this._sourceFileList[i];
if (!this._isFileNeeded(fileInfo)) { if (!this._isFileNeeded(fileInfo)) {
fileDiagnostics.push({ fileDiagnostics.push({
@ -975,33 +985,34 @@ export class Program {
return false; return false;
} }
private _isImportAllowed(importer: SourceFileInfo, importResult: ImportResult, private _isImportAllowed(importer: SourceFileInfo, importResult: ImportResult, isImportStubFile: boolean): boolean {
isImportStubFile: boolean): boolean {
let thirdPartyImportAllowed = this._configOptions.useLibraryCodeForTypes; let thirdPartyImportAllowed = this._configOptions.useLibraryCodeForTypes;
if (importResult.importType === ImportType.ThirdParty || if (
(importer.isThirdPartyImport && importResult.importType === ImportType.Local)) { importResult.importType === ImportType.ThirdParty ||
(importer.isThirdPartyImport && importResult.importType === ImportType.Local)
) {
if (this._allowedThirdPartyImports) { if (this._allowedThirdPartyImports) {
if (importResult.isRelative) { if (importResult.isRelative) {
// If it's a relative import, we'll allow it because the // If it's a relative import, we'll allow it because the
// importer was already deemed to be allowed. // importer was already deemed to be allowed.
thirdPartyImportAllowed = true; thirdPartyImportAllowed = true;
} else if (this._allowedThirdPartyImports.some((importName: string) => { } else if (
// If this import name is the one that was explicitly this._allowedThirdPartyImports.some((importName: string) => {
// allowed or is a child of that import name, // If this import name is the one that was explicitly
// it's considered allowed. // allowed or is a child of that import name,
if (importResult.importName === importName) { // it's considered allowed.
return true; if (importResult.importName === importName) {
} return true;
}
if (importResult.importName.startsWith(importName + '.')) { if (importResult.importName.startsWith(importName + '.')) {
return true; return true;
} }
return false; return false;
})) { })
) {
thirdPartyImportAllowed = true; thirdPartyImportAllowed = true;
} }
} }
@ -1018,9 +1029,7 @@ export class Program {
return true; return true;
} }
private _updateSourceFileImports(sourceFileInfo: SourceFileInfo, private _updateSourceFileImports(sourceFileInfo: SourceFileInfo, options: ConfigOptions): SourceFileInfo[] {
options: ConfigOptions): SourceFileInfo[] {
const filesAdded: SourceFileInfo[] = []; const filesAdded: SourceFileInfo[] = [];
// Get the new list of imports and see if it changed from the last // Get the new list of imports and see if it changed from the last
@ -1033,12 +1042,12 @@ export class Program {
if (importResult.isImportFound) { if (importResult.isImportFound) {
if (this._isImportAllowed(sourceFileInfo, importResult, importResult.isStubFile)) { if (this._isImportAllowed(sourceFileInfo, importResult, importResult.isStubFile)) {
if (importResult.resolvedPaths.length > 0) { if (importResult.resolvedPaths.length > 0) {
const filePath = importResult.resolvedPaths[ const filePath = importResult.resolvedPaths[importResult.resolvedPaths.length - 1];
importResult.resolvedPaths.length - 1];
if (filePath) { if (filePath) {
newImportPathMap.set(filePath, { newImportPathMap.set(filePath, {
isTypeshedFile: !!importResult.isTypeshedFile, isTypeshedFile: !!importResult.isTypeshedFile,
isThirdPartyImport: importResult.importType === ImportType.ThirdParty || isThirdPartyImport:
importResult.importType === ImportType.ThirdParty ||
(sourceFileInfo.isThirdPartyImport && importResult.importType === ImportType.Local) (sourceFileInfo.isThirdPartyImport && importResult.importType === ImportType.Local)
}); });
} }
@ -1049,18 +1058,21 @@ export class Program {
if (this._isImportAllowed(sourceFileInfo, importResult, implicitImport.isStubFile)) { if (this._isImportAllowed(sourceFileInfo, importResult, implicitImport.isStubFile)) {
newImportPathMap.set(implicitImport.path, { newImportPathMap.set(implicitImport.path, {
isTypeshedFile: !!importResult.isTypeshedFile, isTypeshedFile: !!importResult.isTypeshedFile,
isThirdPartyImport: importResult.importType === ImportType.ThirdParty || isThirdPartyImport:
importResult.importType === ImportType.ThirdParty ||
(sourceFileInfo.isThirdPartyImport && importResult.importType === ImportType.Local) (sourceFileInfo.isThirdPartyImport && importResult.importType === ImportType.Local)
}); });
} }
}); });
} else if (options.verboseOutput) { } else if (options.verboseOutput) {
if (!sourceFileInfo.isTypeshedFile || options.diagnosticSettings.reportTypeshedErrors) { if (!sourceFileInfo.isTypeshedFile || options.diagnosticSettings.reportTypeshedErrors) {
this._console.log(`Could not import '${ importResult.importName }' ` + this._console.log(
`in file '${ sourceFileInfo.sourceFile.getFilePath() }'`); `Could not import '${importResult.importName}' ` +
`in file '${sourceFileInfo.sourceFile.getFilePath()}'`
);
if (importResult.importFailureInfo) { if (importResult.importFailureInfo) {
importResult.importFailureInfo.forEach(diag => { importResult.importFailureInfo.forEach(diag => {
this._console.log(` ${ diag }`); this._console.log(` ${diag}`);
}); });
} }
} }
@ -1074,7 +1086,8 @@ export class Program {
// A previous import was removed. // A previous import was removed.
if (!newImportPathMap.has(oldFilePath)) { if (!newImportPathMap.has(oldFilePath)) {
importInfo.importedBy = importInfo.importedBy.filter( importInfo.importedBy = importInfo.importedBy.filter(
fi => fi.sourceFile.getFilePath() !== sourceFileInfo.sourceFile.getFilePath()); fi => fi.sourceFile.getFilePath() !== sourceFileInfo.sourceFile.getFilePath()
);
} else { } else {
updatedImportMap.set(oldFilePath, importInfo); updatedImportMap.set(oldFilePath, importInfo);
} }
@ -1091,8 +1104,11 @@ export class Program {
} else { } else {
const sourceFile = new SourceFile( const sourceFile = new SourceFile(
this._fs, this._fs,
importPath, importInfo.isTypeshedFile, importPath,
importInfo.isThirdPartyImport, this._console); importInfo.isTypeshedFile,
importInfo.isThirdPartyImport,
this._console
);
importedFileInfo = { importedFileInfo = {
sourceFile, sourceFile,
isTracked: false, isTracked: false,
@ -1127,8 +1143,7 @@ export class Program {
sourceFileInfo.builtinsImport = undefined; sourceFileInfo.builtinsImport = undefined;
const builtinsImport = sourceFileInfo.sourceFile.getBuiltinsImport(); const builtinsImport = sourceFileInfo.sourceFile.getBuiltinsImport();
if (builtinsImport) { if (builtinsImport) {
const resolvedBuiltinsPath = builtinsImport.resolvedPaths[ const resolvedBuiltinsPath = builtinsImport.resolvedPaths[builtinsImport.resolvedPaths.length - 1];
builtinsImport.resolvedPaths.length - 1];
sourceFileInfo.builtinsImport = this._sourceFileMap.get(resolvedBuiltinsPath); sourceFileInfo.builtinsImport = this._sourceFileMap.get(resolvedBuiltinsPath);
} }

77
server/src/analyzer/pythonPathUtils.ts
View File

@ -1,18 +1,24 @@
/* /*
* pythonPathUtils.ts * pythonPathUtils.ts
* Copyright (c) Microsoft Corporation. * Copyright (c) Microsoft Corporation.
* Licensed under the MIT license. * Licensed under the MIT license.
* Author: Eric Traut * Author: Eric Traut
* *
* Utility routines used to resolve various paths in python. * Utility routines used to resolve various paths in python.
*/ */
import * as child_process from 'child_process'; import * as child_process from 'child_process';
import { ConfigOptions } from '../common/configOptions'; import { ConfigOptions } from '../common/configOptions';
import * as pathConsts from '../common/pathConsts'; import * as pathConsts from '../common/pathConsts';
import { combinePaths, ensureTrailingDirectorySeparator, getDirectoryPath, import {
getFileSystemEntries, isDirectory, normalizePath } from '../common/pathUtils'; combinePaths,
ensureTrailingDirectorySeparator,
getDirectoryPath,
getFileSystemEntries,
isDirectory,
normalizePath
} from '../common/pathUtils';
import { VirtualFileSystem } from '../common/vfs'; import { VirtualFileSystem } from '../common/vfs';
const cachedSearchPaths = new Map<string, string[]>(); const cachedSearchPaths = new Map<string, string[]>();
@ -23,8 +29,7 @@ export function getTypeShedFallbackPath(fs: VirtualFileSystem) {
return undefined; return undefined;
} }
moduleDirectory = getDirectoryPath(ensureTrailingDirectorySeparator( moduleDirectory = getDirectoryPath(ensureTrailingDirectorySeparator(normalizePath(moduleDirectory)));
normalizePath(moduleDirectory)));
const typeshedPath = combinePaths(moduleDirectory, pathConsts.typeshedFallback); const typeshedPath = combinePaths(moduleDirectory, pathConsts.typeshedFallback);
if (fs.existsSync(typeshedPath)) { if (fs.existsSync(typeshedPath)) {
@ -45,9 +50,12 @@ export function getTypeshedSubdirectory(typeshedPath: string, isStdLib: boolean)
return combinePaths(typeshedPath, isStdLib ? 'stdlib' : 'third_party'); return combinePaths(typeshedPath, isStdLib ? 'stdlib' : 'third_party');
} }
export function findPythonSearchPaths(fs: VirtualFileSystem, configOptions: ConfigOptions, export function findPythonSearchPaths(
venv: string | undefined, importFailureInfo: string[]): string[] | undefined { fs: VirtualFileSystem,
configOptions: ConfigOptions,
venv: string | undefined,
importFailureInfo: string[]
): string[] | undefined {
importFailureInfo.push('Finding python search paths'); importFailureInfo.push('Finding python search paths');
let venvPath: string | undefined; let venvPath: string | undefined;
@ -64,14 +72,14 @@ export function findPythonSearchPaths(fs: VirtualFileSystem, configOptions: Conf
if (venvPath) { if (venvPath) {
let libPath = combinePaths(venvPath, pathConsts.lib); let libPath = combinePaths(venvPath, pathConsts.lib);
if (fs.existsSync(libPath)) { if (fs.existsSync(libPath)) {
importFailureInfo.push(`Found path '${ libPath }'; looking for ${ pathConsts.sitePackages }`); importFailureInfo.push(`Found path '${libPath}'; looking for ${pathConsts.sitePackages}`);
} else { } else {
importFailureInfo.push(`Did not find '${ libPath }'; trying 'Lib' instead`); importFailureInfo.push(`Did not find '${libPath}'; trying 'Lib' instead`);
libPath = combinePaths(venvPath, 'Lib'); libPath = combinePaths(venvPath, 'Lib');
if (fs.existsSync(libPath)) { if (fs.existsSync(libPath)) {
importFailureInfo.push(`Found path '${ libPath }'; looking for ${ pathConsts.sitePackages }`); importFailureInfo.push(`Found path '${libPath}'; looking for ${pathConsts.sitePackages}`);
} else { } else {
importFailureInfo.push(`Did not find '${ libPath }'`); importFailureInfo.push(`Did not find '${libPath}'`);
libPath = ''; libPath = '';
} }
} }
@ -79,10 +87,10 @@ export function findPythonSearchPaths(fs: VirtualFileSystem, configOptions: Conf
if (libPath) { if (libPath) {
const sitePackagesPath = combinePaths(libPath, pathConsts.sitePackages); const sitePackagesPath = combinePaths(libPath, pathConsts.sitePackages);
if (fs.existsSync(sitePackagesPath)) { if (fs.existsSync(sitePackagesPath)) {
importFailureInfo.push(`Found path '${ sitePackagesPath }'`); importFailureInfo.push(`Found path '${sitePackagesPath}'`);
return [sitePackagesPath]; return [sitePackagesPath];
} else { } else {
importFailureInfo.push(`Did not find '${ sitePackagesPath }', so looking for python subdirectory`); importFailureInfo.push(`Did not find '${sitePackagesPath}', so looking for python subdirectory`);
} }
// We didn't find a site-packages directory directly in the lib // We didn't find a site-packages directory directly in the lib
@ -93,26 +101,27 @@ export function findPythonSearchPaths(fs: VirtualFileSystem, configOptions: Conf
if (dirName.startsWith('python')) { if (dirName.startsWith('python')) {
const dirPath = combinePaths(libPath, dirName, pathConsts.sitePackages); const dirPath = combinePaths(libPath, dirName, pathConsts.sitePackages);
if (fs.existsSync(dirPath)) { if (fs.existsSync(dirPath)) {
importFailureInfo.push(`Found path '${ dirPath }'`); importFailureInfo.push(`Found path '${dirPath}'`);
return [dirPath]; return [dirPath];
} else { } else {
importFailureInfo.push(`Path '${ dirPath }' is not a valid directory`); importFailureInfo.push(`Path '${dirPath}' is not a valid directory`);
} }
} }
} }
} }
importFailureInfo.push(`Did not find '${ pathConsts.sitePackages }'. Falling back on python interpreter.`); importFailureInfo.push(`Did not find '${pathConsts.sitePackages}'. Falling back on python interpreter.`);
} }
// Fall back on the python interpreter. // Fall back on the python interpreter.
return getPythonPathFromPythonInterpreter(fs, configOptions.pythonPath, importFailureInfo); return getPythonPathFromPythonInterpreter(fs, configOptions.pythonPath, importFailureInfo);
} }
export function getPythonPathFromPythonInterpreter(fs: VirtualFileSystem, export function getPythonPathFromPythonInterpreter(
fs: VirtualFileSystem,
interpreterPath: string | undefined, interpreterPath: string | undefined,
importFailureInfo: string[]): string[] { importFailureInfo: string[]
): string[] {
const searchKey = interpreterPath || ''; const searchKey = interpreterPath || '';
// If we've seen this request before, return the cached results. // If we've seen this request before, return the cached results.
@ -136,13 +145,11 @@ export function getPythonPathFromPythonInterpreter(fs: VirtualFileSystem,
let execOutput: string; let execOutput: string;
if (interpreterPath) { if (interpreterPath) {
importFailureInfo.push(`Executing interpreter at '${ interpreterPath }'`); importFailureInfo.push(`Executing interpreter at '${interpreterPath}'`);
execOutput = child_process.execFileSync( execOutput = child_process.execFileSync(interpreterPath, commandLineArgs, { encoding: 'utf8' });
interpreterPath, commandLineArgs, { encoding: 'utf8' });
} else { } else {
importFailureInfo.push(`Executing python interpreter`); importFailureInfo.push(`Executing python interpreter`);
execOutput = child_process.execFileSync( execOutput = child_process.execFileSync('python', commandLineArgs, { encoding: 'utf8' });
'python', commandLineArgs, { encoding: 'utf8' });
} }
// Parse the execOutput. It should be a JSON-encoded array of paths. // Parse the execOutput. It should be a JSON-encoded array of paths.
@ -157,7 +164,7 @@ export function getPythonPathFromPythonInterpreter(fs: VirtualFileSystem,
if (fs.existsSync(normalizedPath) && isDirectory(fs, normalizedPath)) { if (fs.existsSync(normalizedPath) && isDirectory(fs, normalizedPath)) {
pythonPaths.push(normalizedPath); pythonPaths.push(normalizedPath);
} else { } else {
importFailureInfo.push(`Skipping '${ normalizedPath }' because it is not a valid directory`); importFailureInfo.push(`Skipping '${normalizedPath}' because it is not a valid directory`);
} }
} }
} }
@ -166,7 +173,7 @@ export function getPythonPathFromPythonInterpreter(fs: VirtualFileSystem,
importFailureInfo.push(`Found no valid directories`); importFailureInfo.push(`Found no valid directories`);
} }
} catch (err) { } catch (err) {
importFailureInfo.push(`Could not parse output: '${ execOutput }'`); importFailureInfo.push(`Could not parse output: '${execOutput}'`);
throw err; throw err;
} }
} catch { } catch {
@ -174,9 +181,9 @@ export function getPythonPathFromPythonInterpreter(fs: VirtualFileSystem,
} }
cachedSearchPaths.set(searchKey, pythonPaths); cachedSearchPaths.set(searchKey, pythonPaths);
importFailureInfo.push(`Received ${ pythonPaths.length } paths from interpreter`); importFailureInfo.push(`Received ${pythonPaths.length} paths from interpreter`);
pythonPaths.forEach(path => { pythonPaths.forEach(path => {
importFailureInfo.push(` ${ path }`); importFailureInfo.push(` ${path}`);
}); });
return pythonPaths; return pythonPaths;
} }

32
server/src/analyzer/scope.ts
View File

@ -1,13 +1,13 @@
/* /*
* scope.ts * scope.ts
* Copyright (c) Microsoft Corporation. * Copyright (c) Microsoft Corporation.
* Licensed under the MIT license. * Licensed under the MIT license.
* Author: Eric Traut * Author: Eric Traut
* *
* Represents an evaluation scope and its defined symbols. * Represents an evaluation scope and its defined symbols.
* It also contains a link to a parent scope (except for the * It also contains a link to a parent scope (except for the
* top-most built-in scope). * top-most built-in scope).
*/ */
import { fail } from '../common/debug'; import { fail } from '../common/debug';
import { Symbol, SymbolFlags, SymbolTable } from './symbol'; import { Symbol, SymbolFlags, SymbolTable } from './symbol';
@ -103,9 +103,11 @@ export class Scope {
return symbol; return symbol;
} }
private _lookUpSymbolRecursiveInternal(name: string, isOutsideCallerModule: boolean, private _lookUpSymbolRecursiveInternal(
isBeyondExecutionScope: boolean): SymbolWithScope | undefined { name: string,
isOutsideCallerModule: boolean,
isBeyondExecutionScope: boolean
): SymbolWithScope | undefined {
const symbol = this.symbolTable.get(name); const symbol = this.symbolTable.get(name);
if (symbol) { if (symbol) {
@ -127,9 +129,11 @@ export class Scope {
// If our recursion is about to take us outside the scope of the current // If our recursion is about to take us outside the scope of the current
// module (i.e. into a built-in scope), indicate as such with the second // module (i.e. into a built-in scope), indicate as such with the second
// parameter. // parameter.
return this.parent._lookUpSymbolRecursiveInternal(name, return this.parent._lookUpSymbolRecursiveInternal(
name,
isOutsideCallerModule || this.type === ScopeType.Module, isOutsideCallerModule || this.type === ScopeType.Module,
isBeyondExecutionScope || this.isIndependentlyExecutable()); isBeyondExecutionScope || this.isIndependentlyExecutable()
);
} }
return undefined; return undefined;

16
server/src/analyzer/scopeUtils.ts
View File

@ -1,12 +1,12 @@
/* /*
* scopeUtils.ts * scopeUtils.ts
* Copyright (c) Microsoft Corporation. * Copyright (c) Microsoft Corporation.
* Licensed under the MIT license. * Licensed under the MIT license.
* Author: Eric Traut * Author: Eric Traut
* *
* Static utility methods related to scopes and their related * Static utility methods related to scopes and their related
* symbol tables. * symbol tables.
*/ */
import { ParseNode } from '../parser/parseNodes'; import { ParseNode } from '../parser/parseNodes';
import { getScope } from './analyzerNodeInfo'; import { getScope } from './analyzerNodeInfo';

219
server/src/analyzer/service.ts
View File

@ -1,12 +1,12 @@
/* /*
* service.ts * service.ts
* Copyright (c) Microsoft Corporation. * Copyright (c) Microsoft Corporation.
* Licensed under the MIT license. * Licensed under the MIT license.
* Author: Eric Traut * Author: Eric Traut
* *
* A persistent service that is able to analyze a collection of * A persistent service that is able to analyze a collection of
* Python files. * Python files.
*/ */
import { CompletionItem, CompletionList, DocumentSymbol, SymbolInformation } from 'vscode-languageserver'; import { CompletionItem, CompletionList, DocumentSymbol, SymbolInformation } from 'vscode-languageserver';
@ -18,9 +18,16 @@ import { Diagnostic } from '../common/diagnostic';
import { FileDiagnostics } from '../common/diagnosticSink'; import { FileDiagnostics } from '../common/diagnosticSink';
import { FileEditAction, TextEditAction } from '../common/editAction'; import { FileEditAction, TextEditAction } from '../common/editAction';
import { import {
combinePaths, FileSpec, forEachAncestorDirectory, getDirectoryPath, combinePaths,
getFileName, getFileSpec, getFileSystemEntries, isDirectory, FileSpec,
normalizePath, stripFileExtension forEachAncestorDirectory,
getDirectoryPath,
getFileName,
getFileSpec,
getFileSystemEntries,
isDirectory,
normalizePath,
stripFileExtension
} from '../common/pathUtils'; } from '../common/pathUtils';
import { DocumentRange, Position, Range } from '../common/textRange'; import { DocumentRange, Position, Range } from '../common/textRange';
import { Duration, timingStats } from '../common/timing'; import { Duration, timingStats } from '../common/timing';
@ -70,9 +77,13 @@ export class AnalyzerService {
private _requireTrackedFileUpdate = true; private _requireTrackedFileUpdate = true;
private _lastUserInteractionTime = Date.now(); private _lastUserInteractionTime = Date.now();
constructor(instanceName: string, fs: VirtualFileSystem, console?: ConsoleInterface, constructor(
importResolverFactory?: ImportResolverFactory, configOptions?: ConfigOptions) { instanceName: string,
fs: VirtualFileSystem,
console?: ConsoleInterface,
importResolverFactory?: ImportResolverFactory,
configOptions?: ConfigOptions
) {
this._instanceName = instanceName; this._instanceName = instanceName;
this._console = console || new StandardConsole(); this._console = console || new StandardConsole();
this._configOptions = configOptions ?? new ConfigOptions(process.cwd()); this._configOptions = configOptions ?? new ConfigOptions(process.cwd());
@ -84,7 +95,13 @@ export class AnalyzerService {
} }
clone(instanceName: string): AnalyzerService { clone(instanceName: string): AnalyzerService {
return new AnalyzerService(instanceName, this._fs, this._console, this._importResolverFactory, this._configOptions); return new AnalyzerService(
instanceName,
this._fs,
this._console,
this._importResolverFactory,
this._configOptions
);
} }
dispose() { dispose() {
@ -113,8 +130,9 @@ export class AnalyzerService {
this._program.setConfigOptions(this._configOptions); this._program.setConfigOptions(this._configOptions);
this._typeStubTargetImportName = commandLineOptions.typeStubTargetImportName; this._typeStubTargetImportName = commandLineOptions.typeStubTargetImportName;
this._executionRootPath = normalizePath(combinePaths( this._executionRootPath = normalizePath(
commandLineOptions.executionRoot, this._configOptions.projectRoot)); combinePaths(commandLineOptions.executionRoot, this._configOptions.projectRoot)
);
this._applyConfigOptions(); this._applyConfigOptions();
} }
@ -135,15 +153,15 @@ export class AnalyzerService {
this._scheduleReanalysis(false); this._scheduleReanalysis(false);
} }
getDefinitionForPosition(filePath: string, position: Position): getDefinitionForPosition(filePath: string, position: Position): DocumentRange[] | undefined {
DocumentRange[] | undefined {
return this._program.getDefinitionsForPosition(filePath, position); return this._program.getDefinitionsForPosition(filePath, position);
} }
getReferencesForPosition(filePath: string, position: Position, getReferencesForPosition(
includeDeclaration: boolean): DocumentRange[] | undefined { filePath: string,
position: Position,
includeDeclaration: boolean
): DocumentRange[] | undefined {
return this._program.getReferencesForPosition(filePath, position, includeDeclaration); return this._program.getReferencesForPosition(filePath, position, includeDeclaration);
} }
@ -155,21 +173,15 @@ export class AnalyzerService {
this._program.addSymbolsForWorkspace(symbolList, query); this._program.addSymbolsForWorkspace(symbolList, query);
} }
getHoverForPosition(filePath: string, position: Position): getHoverForPosition(filePath: string, position: Position): HoverResults | undefined {
HoverResults | undefined {
return this._program.getHoverForPosition(filePath, position); return this._program.getHoverForPosition(filePath, position);
} }
getSignatureHelpForPosition(filePath: string, position: Position): getSignatureHelpForPosition(filePath: string, position: Position): SignatureHelpResults | undefined {
SignatureHelpResults | undefined {
return this._program.getSignatureHelpForPosition(filePath, position); return this._program.getSignatureHelpForPosition(filePath, position);
} }
getCompletionsForPosition(filePath: string, position: Position, getCompletionsForPosition(filePath: string, position: Position, workspacePath: string): CompletionList | undefined {
workspacePath: string): CompletionList | undefined {
return this._program.getCompletionsForPosition(filePath, position, workspacePath); return this._program.getCompletionsForPosition(filePath, position, workspacePath);
} }
@ -181,9 +193,7 @@ export class AnalyzerService {
return this._program.performQuickAction(filePath, command, args); return this._program.performQuickAction(filePath, command, args);
} }
renameSymbolAtPosition(filePath: string, position: Position, renameSymbolAtPosition(filePath: string, position: Position, newName: string): FileEditAction[] | undefined {
newName: string): FileEditAction[] | undefined {
return this._program.renameSymbolAtPosition(filePath, position, newName); return this._program.renameSymbolAtPosition(filePath, position, newName);
} }
@ -251,10 +261,12 @@ export class AnalyzerService {
// If the config file path was specified, determine whether it's // If the config file path was specified, determine whether it's
// a directory (in which case the default config file name is assumed) // a directory (in which case the default config file name is assumed)
// or a file. // or a file.
configFilePath = combinePaths(commandLineOptions.executionRoot, configFilePath = combinePaths(
normalizePath(commandLineOptions.configFilePath)); commandLineOptions.executionRoot,
normalizePath(commandLineOptions.configFilePath)
);
if (!this._fs.existsSync(configFilePath)) { if (!this._fs.existsSync(configFilePath)) {
this._console.log(`Configuration file not found at ${ configFilePath }.`); this._console.log(`Configuration file not found at ${configFilePath}.`);
configFilePath = commandLineOptions.executionRoot; configFilePath = commandLineOptions.executionRoot;
} else { } else {
if (configFilePath.toLowerCase().endsWith('.json')) { if (configFilePath.toLowerCase().endsWith('.json')) {
@ -263,7 +275,7 @@ export class AnalyzerService {
projectRoot = configFilePath; projectRoot = configFilePath;
configFilePath = this._findConfigFile(configFilePath); configFilePath = this._findConfigFile(configFilePath);
if (!configFilePath) { if (!configFilePath) {
this._console.log(`Configuration file not found at ${ projectRoot }.`); this._console.log(`Configuration file not found at ${projectRoot}.`);
} }
} }
} }
@ -302,7 +314,7 @@ export class AnalyzerService {
// If we found a config file, parse it to compute the effective options. // If we found a config file, parse it to compute the effective options.
if (configFilePath) { if (configFilePath) {
this._console.log(`Loading configuration file at ${ configFilePath }`); this._console.log(`Loading configuration file at ${configFilePath}`);
const configJsonObj = this._parseConfigFile(configFilePath); const configJsonObj = this._parseConfigFile(configFilePath);
if (configJsonObj) { if (configJsonObj) {
configOptions.initializeFromJson(configJsonObj, this._console); configOptions.initializeFromJson(configJsonObj, this._console);
@ -312,14 +324,14 @@ export class AnalyzerService {
// If no include paths were provided, assume that all files within // If no include paths were provided, assume that all files within
// the project should be included. // the project should be included.
if (configOptions.include.length === 0) { if (configOptions.include.length === 0) {
this._console.log(`No include entries specified; assuming ${ configFilePath }`); this._console.log(`No include entries specified; assuming ${configFilePath}`);
configOptions.include.push(getFileSpec(configFileDir, '.')); configOptions.include.push(getFileSpec(configFileDir, '.'));
} }
// If there was no explicit set of excludes, add a few common ones to avoid long scan times. // If there was no explicit set of excludes, add a few common ones to avoid long scan times.
if (configOptions.exclude.length === 0) { if (configOptions.exclude.length === 0) {
defaultExcludes.forEach(exclude => { defaultExcludes.forEach(exclude => {
this._console.log(`Auto-excluding ${ exclude }`); this._console.log(`Auto-excluding ${exclude}`);
configOptions.exclude.push(getFileSpec(configFileDir, exclude)); configOptions.exclude.push(getFileSpec(configFileDir, exclude));
}); });
} }
@ -328,12 +340,14 @@ export class AnalyzerService {
} }
const reportDuplicateSetting = (settingName: string) => { const reportDuplicateSetting = (settingName: string) => {
const settingSource = commandLineOptions.fromVsCodeExtension ? const settingSource = commandLineOptions.fromVsCodeExtension
'the VS Code settings' : 'a command-line option'; ? 'the VS Code settings'
: 'a command-line option';
this._console.log( this._console.log(
`The ${ settingName } has been specified in both the config file and ` + `The ${settingName} has been specified in both the config file and ` +
`${ settingSource }. The value in the config file (${ configOptions.venvPath }) ` + `${settingSource}. The value in the config file (${configOptions.venvPath}) ` +
`will take precedence`); `will take precedence`
);
}; };
// Apply the command-line options if the corresponding // Apply the command-line options if the corresponding
@ -348,8 +362,9 @@ export class AnalyzerService {
} }
if (commandLineOptions.pythonPath) { if (commandLineOptions.pythonPath) {
this._console.log(`Setting pythonPath for service "${ this._instanceName }": ` + this._console.log(
`"${ commandLineOptions.pythonPath }"`); `Setting pythonPath for service "${this._instanceName}": ` + `"${commandLineOptions.pythonPath}"`
);
configOptions.pythonPath = commandLineOptions.pythonPath; configOptions.pythonPath = commandLineOptions.pythonPath;
} }
@ -374,8 +389,7 @@ export class AnalyzerService {
// or inconsistent information. // or inconsistent information.
if (configOptions.venvPath) { if (configOptions.venvPath) {
if (!this._fs.existsSync(configOptions.venvPath) || !isDirectory(this._fs, configOptions.venvPath)) { if (!this._fs.existsSync(configOptions.venvPath) || !isDirectory(this._fs, configOptions.venvPath)) {
this._console.log( this._console.log(`venvPath ${configOptions.venvPath} is not a valid directory.`);
`venvPath ${ configOptions.venvPath } is not a valid directory.`);
} }
// venvPath without defaultVenv means it won't do anything while resolveImport. // venvPath without defaultVenv means it won't do anything while resolveImport.
@ -387,18 +401,20 @@ export class AnalyzerService {
if (!this._fs.existsSync(fullVenvPath) || !isDirectory(this._fs, fullVenvPath)) { if (!this._fs.existsSync(fullVenvPath) || !isDirectory(this._fs, fullVenvPath)) {
this._console.log( this._console.log(
`venv ${ configOptions.defaultVenv } subdirectory not found ` + `venv ${configOptions.defaultVenv} subdirectory not found ` +
`in venv path ${ configOptions.venvPath }.`); `in venv path ${configOptions.venvPath}.`
);
} else { } else {
const importFailureInfo: string[] = []; const importFailureInfo: string[] = [];
if (findPythonSearchPaths(this._fs, configOptions, undefined, importFailureInfo) === undefined) { if (findPythonSearchPaths(this._fs, configOptions, undefined, importFailureInfo) === undefined) {
this._console.log( this._console.log(
`site-packages directory cannot be located for venvPath ` + `site-packages directory cannot be located for venvPath ` +
`${ configOptions.venvPath } and venv ${ configOptions.defaultVenv }.`); `${configOptions.venvPath} and venv ${configOptions.defaultVenv}.`
);
if (configOptions.verboseOutput) { if (configOptions.verboseOutput) {
importFailureInfo.forEach(diag => { importFailureInfo.forEach(diag => {
this._console.log(` ${ diag }`); this._console.log(` ${diag}`);
}); });
} }
} }
@ -406,28 +422,29 @@ export class AnalyzerService {
} }
} else { } else {
const importFailureInfo: string[] = []; const importFailureInfo: string[] = [];
const pythonPaths = getPythonPathFromPythonInterpreter(this._fs, configOptions.pythonPath, importFailureInfo); const pythonPaths = getPythonPathFromPythonInterpreter(
this._fs,
configOptions.pythonPath,
importFailureInfo
);
if (pythonPaths.length === 0) { if (pythonPaths.length === 0) {
if (configOptions.verboseOutput) { if (configOptions.verboseOutput) {
this._console.log( this._console.log(`No search paths found for configured python interpreter.`);
`No search paths found for configured python interpreter.`);
} }
} else { } else {
if (configOptions.verboseOutput) { if (configOptions.verboseOutput) {
this._console.log( this._console.log(`Search paths found for configured python interpreter:`);
`Search paths found for configured python interpreter:`);
pythonPaths.forEach(path => { pythonPaths.forEach(path => {
this._console.log(` ${ path }`); this._console.log(` ${path}`);
}); });
} }
} }
if (configOptions.verboseOutput) { if (configOptions.verboseOutput) {
if (importFailureInfo.length > 0) { if (importFailureInfo.length > 0) {
this._console.log( this._console.log(`When attempting to get search paths from python interpreter:`);
`When attempting to get search paths from python interpreter:`);
importFailureInfo.forEach(diag => { importFailureInfo.forEach(diag => {
this._console.log(` ${ diag }`); this._console.log(` ${diag}`);
}); });
} }
} }
@ -436,22 +453,22 @@ export class AnalyzerService {
// Is there a reference to a venv? If so, there needs to be a valid venvPath. // Is there a reference to a venv? If so, there needs to be a valid venvPath.
if (configOptions.defaultVenv || configOptions.executionEnvironments.find(e => !!e.venv)) { if (configOptions.defaultVenv || configOptions.executionEnvironments.find(e => !!e.venv)) {
if (!configOptions.venvPath) { if (!configOptions.venvPath) {
this._console.log( this._console.log(`venvPath not specified, so venv settings will be ignored.`);
`venvPath not specified, so venv settings will be ignored.`);
} }
} }
if (configOptions.typeshedPath) { if (configOptions.typeshedPath) {
if (!this._fs.existsSync(configOptions.typeshedPath) || !isDirectory(this._fs, configOptions.typeshedPath)) { if (
this._console.log( !this._fs.existsSync(configOptions.typeshedPath) ||
`typeshedPath ${ configOptions.typeshedPath } is not a valid directory.`); !isDirectory(this._fs, configOptions.typeshedPath)
) {
this._console.log(`typeshedPath ${configOptions.typeshedPath} is not a valid directory.`);
} }
} }
if (configOptions.typingsPath) { if (configOptions.typingsPath) {
if (!this._fs.existsSync(configOptions.typingsPath) || !isDirectory(this._fs, configOptions.typingsPath)) { if (!this._fs.existsSync(configOptions.typingsPath) || !isDirectory(this._fs, configOptions.typingsPath)) {
this._console.log( this._console.log(`typingsPath ${configOptions.typingsPath} is not a valid directory.`);
`typingsPath ${ configOptions.typingsPath } is not a valid directory.`);
} }
} }
@ -461,8 +478,7 @@ export class AnalyzerService {
writeTypeStub() { writeTypeStub() {
const typingsPath = this._configOptions.typingsPath; const typingsPath = this._configOptions.typingsPath;
if (!this._typeStubTargetPath || !this._typeStubTargetImportName) { if (!this._typeStubTargetPath || !this._typeStubTargetImportName) {
const errMsg = `Import '${ this._typeStubTargetImportName }'` + const errMsg = `Import '${this._typeStubTargetImportName}'` + ` could not be resolved`;
` could not be resolved`;
this._console.error(errMsg); this._console.error(errMsg);
throw new Error(errMsg); throw new Error(errMsg);
} }
@ -479,8 +495,7 @@ export class AnalyzerService {
if (typeStubInputTargetParts[0].length === 0) { if (typeStubInputTargetParts[0].length === 0) {
// We should never get here because the import resolution // We should never get here because the import resolution
// would have failed. // would have failed.
const errMsg = `Import '${ this._typeStubTargetImportName }'` + const errMsg = `Import '${this._typeStubTargetImportName}'` + ` could not be resolved`;
` could not be resolved`;
this._console.error(errMsg); this._console.error(errMsg);
throw new Error(errMsg); throw new Error(errMsg);
} }
@ -491,7 +506,7 @@ export class AnalyzerService {
this._fs.mkdirSync(typingsPath); this._fs.mkdirSync(typingsPath);
} }
} catch (e) { } catch (e) {
const errMsg = `Could not create typings directory '${ typingsPath }'`; const errMsg = `Could not create typings directory '${typingsPath}'`;
this._console.error(errMsg); this._console.error(errMsg);
throw new Error(errMsg); throw new Error(errMsg);
} }
@ -504,7 +519,7 @@ export class AnalyzerService {
this._fs.mkdirSync(typingsSubdirPath); this._fs.mkdirSync(typingsSubdirPath);
} }
} catch (e) { } catch (e) {
const errMsg = `Could not create typings subdirectory '${ typingsSubdirPath }'`; const errMsg = `Could not create typings subdirectory '${typingsSubdirPath}'`;
this._console.error(errMsg); this._console.error(errMsg);
throw new Error(errMsg); throw new Error(errMsg);
} }
@ -551,7 +566,7 @@ export class AnalyzerService {
try { try {
configContents = this._fs.readFileSync(configPath, 'utf8'); configContents = this._fs.readFileSync(configPath, 'utf8');
} catch { } catch {
this._console.log(`Config file "${ configPath }" could not be read.`); this._console.log(`Config file "${configPath}" could not be read.`);
this._reportConfigParseError(); this._reportConfigParseError();
return undefined; return undefined;
} }
@ -575,7 +590,7 @@ export class AnalyzerService {
// resulting in parse errors. We'll give it a little more time and // resulting in parse errors. We'll give it a little more time and
// try again. // try again.
if (parseAttemptCount++ >= 5) { if (parseAttemptCount++ >= 5) {
this._console.log(`Config file "${ configPath }" could not be parsed. Verify that JSON is correct.`); this._console.log(`Config file "${configPath}" could not be parsed. Verify that JSON is correct.`);
this._reportConfigParseError(); this._reportConfigParseError();
return undefined; return undefined;
} }
@ -587,8 +602,7 @@ export class AnalyzerService {
const fileMap = new Map<string, string>(); const fileMap = new Map<string, string>();
timingStats.findFilesTime.timeOperation(() => { timingStats.findFilesTime.timeOperation(() => {
const matchedFiles = this._matchFiles(this._configOptions.include, const matchedFiles = this._matchFiles(this._configOptions.include, this._configOptions.exclude);
this._configOptions.exclude);
for (const file of matchedFiles) { for (const file of matchedFiles) {
fileMap.set(file, file); fileMap.set(file, file);
@ -614,16 +628,14 @@ export class AnalyzerService {
importedSymbols: [] importedSymbols: []
}; };
const importResult = this._importResolver.resolveImport( const importResult = this._importResolver.resolveImport('', execEnv, moduleDescriptor);
'', execEnv, moduleDescriptor);
if (importResult.isImportFound) { if (importResult.isImportFound) {
const filesToImport: string[] = []; const filesToImport: string[] = [];
// Namespace packages resolve to a directory name, so // Namespace packages resolve to a directory name, so
// don't include those. // don't include those.
const resolvedPath = importResult.resolvedPaths[ const resolvedPath = importResult.resolvedPaths[importResult.resolvedPaths.length - 1];
importResult.resolvedPaths.length - 1];
// Get the directory that contains the root package. // Get the directory that contains the root package.
let targetPath = getDirectoryPath(resolvedPath); let targetPath = getDirectoryPath(resolvedPath);
@ -647,7 +659,8 @@ export class AnalyzerService {
this._typeStubTargetIsSingleFile = false; this._typeStubTargetIsSingleFile = false;
} else { } else {
filesToImport.push(resolvedPath); filesToImport.push(resolvedPath);
this._typeStubTargetIsSingleFile = importResult.resolvedPaths.length === 1 && this._typeStubTargetIsSingleFile =
importResult.resolvedPaths.length === 1 &&
stripFileExtension(getFileName(importResult.resolvedPaths[0])) !== '__init__'; stripFileExtension(getFileName(importResult.resolvedPaths[0])) !== '__init__';
} }
@ -659,7 +672,7 @@ export class AnalyzerService {
this._program.setAllowedThirdPartyImports([this._typeStubTargetImportName]); this._program.setAllowedThirdPartyImports([this._typeStubTargetImportName]);
this._program.setTrackedFiles(filesToImport); this._program.setTrackedFiles(filesToImport);
} else { } else {
this._console.log(`Import '${ this._typeStubTargetImportName }' not found`); this._console.log(`Import '${this._typeStubTargetImportName}' not found`);
} }
} else { } else {
let fileList: string[] = []; let fileList: string[] = [];
@ -673,8 +686,7 @@ export class AnalyzerService {
if (fileList.length === 0) { if (fileList.length === 0) {
this._console.log(`No source files found.`); this._console.log(`No source files found.`);
} else { } else {
this._console.log(`Found ${ fileList.length } ` + this._console.log(`Found ${fileList.length} ` + `source ${fileList.length === 1 ? 'file' : 'files'}`);
`source ${ fileList.length === 1 ? 'file' : 'files' }`);
} }
} }
@ -733,7 +745,7 @@ export class AnalyzerService {
} }
if (!foundFileSpec) { if (!foundFileSpec) {
this._console.log(`File or directory "${ includeSpec.wildcardRoot }" does not exist.`); this._console.log(`File or directory "${includeSpec.wildcardRoot}" does not exist.`);
} }
}); });
@ -766,12 +778,12 @@ export class AnalyzerService {
try { try {
if (this._verboseOutput) { if (this._verboseOutput) {
this._console.log(`Adding fs watcher for directories:\n ${ fileList.join('\n') }`); this._console.log(`Adding fs watcher for directories:\n ${fileList.join('\n')}`);
} }
this._sourceFileWatcher = this._fs.createFileSystemWatcher(fileList, 'all', (event, path) => { this._sourceFileWatcher = this._fs.createFileSystemWatcher(fileList, 'all', (event, path) => {
if (this._verboseOutput) { if (this._verboseOutput) {
this._console.log(`Received fs event '${ event }' for path '${ path }'`); this._console.log(`Received fs event '${event}' for path '${path}'`);
} }
if (event === 'change') { if (event === 'change') {
@ -782,7 +794,7 @@ export class AnalyzerService {
} }
}); });
} catch { } catch {
this._console.log(`Exception caught when installing fs watcher for:\n ${ fileList.join('\n') }`); this._console.log(`Exception caught when installing fs watcher for:\n ${fileList.join('\n')}`);
} }
} }
} }
@ -798,13 +810,12 @@ export class AnalyzerService {
this._removeConfigFileWatcher(); this._removeConfigFileWatcher();
if (this._configFilePath) { if (this._configFilePath) {
this._configFileWatcher = this._fs.createFileSystemWatcher([this._configFilePath], this._configFileWatcher = this._fs.createFileSystemWatcher([this._configFilePath], 'all', event => {
'all', event => { if (this._verboseOutput) {
if (this._verboseOutput) { this._console.log(`Received fs event '${event}' for config file`);
this._console.log(`Received fs event '${ event }' for config file`); }
} this._scheduleReloadConfigFile();
this._scheduleReloadConfigFile(); });
});
} }
} }
@ -832,7 +843,7 @@ export class AnalyzerService {
if (this._configFilePath) { if (this._configFilePath) {
this._updateConfigFileWatcher(); this._updateConfigFileWatcher();
this._console.log(`Reloading configuration file at ${ this._configFilePath }`); this._console.log(`Reloading configuration file at ${this._configFilePath}`);
const configJsonObj = this._parseConfigFile(this._configFilePath); const configJsonObj = this._parseConfigFile(this._configFilePath);
if (configJsonObj) { if (configJsonObj) {
this._configOptions.initializeFromJson(configJsonObj, this._console); this._configOptions.initializeFromJson(configJsonObj, this._console);
@ -881,7 +892,8 @@ export class AnalyzerService {
const timeUntilNextAnalysisInMs = Math.max( const timeUntilNextAnalysisInMs = Math.max(
minBackoffTimeInMs - timeSinceLastUserInteractionInMs, minBackoffTimeInMs - timeSinceLastUserInteractionInMs,
minTimeBetweenAnalysisPassesInMs); minTimeBetweenAnalysisPassesInMs
);
// Schedule a new timer. // Schedule a new timer.
this._analyzeTimer = setTimeout(() => { this._analyzeTimer = setTimeout(() => {
@ -943,7 +955,8 @@ export class AnalyzerService {
} }
} }
} catch (e) { } catch (e) {
const message: string = (e.stack ? e.stack.toString() : undefined) || const message: string =
(e.stack ? e.stack.toString() : undefined) ||
(typeof e.message === 'string' ? e.message : undefined) || (typeof e.message === 'string' ? e.message : undefined) ||
JSON.stringify(e); JSON.stringify(e);
this._console.log('Error performing analysis: ' + message); this._console.log('Error performing analysis: ' + message);

327
server/src/analyzer/sourceFile.ts
View File

@ -1,18 +1,15 @@
/* /*
* sourceFile.ts * sourceFile.ts
* Copyright (c) Microsoft Corporation. * Copyright (c) Microsoft Corporation.
* Licensed under the MIT license. * Licensed under the MIT license.
* Author: Eric Traut * Author: Eric Traut
* *
* Class that represents a single python source file. * Class that represents a single python source file.
*/ */
import { CompletionItem, CompletionList, DocumentSymbol, SymbolInformation } from 'vscode-languageserver'; import { CompletionItem, CompletionList, DocumentSymbol, SymbolInformation } from 'vscode-languageserver';
import { import { ConfigOptions, ExecutionEnvironment, getDefaultDiagnosticSettings } from '../common/configOptions';
ConfigOptions, ExecutionEnvironment,
getDefaultDiagnosticSettings
} from '../common/configOptions';
import { ConsoleInterface, StandardConsole } from '../common/console'; import { ConsoleInterface, StandardConsole } from '../common/console';
import { assert } from '../common/debug'; import { assert } from '../common/debug';
import { Diagnostic, DiagnosticCategory } from '../common/diagnostic'; import { Diagnostic, DiagnosticCategory } from '../common/diagnostic';
@ -142,9 +139,13 @@ export class SourceFile {
readonly fileSystem: VirtualFileSystem; readonly fileSystem: VirtualFileSystem;
constructor(fs: VirtualFileSystem, filePath: string, isTypeshedStubFile: boolean, constructor(
isThirdPartyImport: boolean, console?: ConsoleInterface) { fs: VirtualFileSystem,
filePath: string,
isTypeshedStubFile: boolean,
isThirdPartyImport: boolean,
console?: ConsoleInterface
) {
this.fileSystem = fs; this.fileSystem = fs;
this._console = console || new StandardConsole(); this._console = console || new StandardConsole();
this._filePath = filePath; this._filePath = filePath;
@ -152,18 +153,19 @@ export class SourceFile {
this._isTypeshedStubFile = isTypeshedStubFile; this._isTypeshedStubFile = isTypeshedStubFile;
this._isThirdPartyImport = isThirdPartyImport; this._isThirdPartyImport = isThirdPartyImport;
const fileName = getFileName(filePath); const fileName = getFileName(filePath);
this._isTypingStubFile = this._isStubFile && ( this._isTypingStubFile =
fileName === 'typing.pyi' || fileName === 'typing_extensions.pyi'); this._isStubFile && (fileName === 'typing.pyi' || fileName === 'typing_extensions.pyi');
this._isBuiltInStubFile = false; this._isBuiltInStubFile = false;
if (this._isStubFile) { if (this._isStubFile) {
if (this._filePath.endsWith(normalizeSlashes('/collections/__init__.pyi')) || if (
this._filePath.endsWith(normalizeSlashes('/collections/__init__.pyi')) ||
fileName === 'builtins.pyi' || fileName === 'builtins.pyi' ||
fileName === '_importlib_modulespec.pyi' || fileName === '_importlib_modulespec.pyi' ||
fileName === 'dataclasses.pyi' || fileName === 'dataclasses.pyi' ||
fileName === 'abc.pyi' || fileName === 'abc.pyi' ||
fileName === 'enum.pyi') { fileName === 'enum.pyi'
) {
this._isBuiltInStubFile = true; this._isBuiltInStubFile = true;
} }
} }
@ -184,9 +186,7 @@ export class SourceFile {
// Returns a list of cached diagnostics from the latest analysis job. // Returns a list of cached diagnostics from the latest analysis job.
// If the prevVersion is specified, the method returns undefined if // If the prevVersion is specified, the method returns undefined if
// the diagnostics haven't changed. // the diagnostics haven't changed.
getDiagnostics(options: ConfigOptions, prevDiagnosticVersion?: number): getDiagnostics(options: ConfigOptions, prevDiagnosticVersion?: number): Diagnostic[] | undefined {
Diagnostic[] | undefined {
if (this._diagnosticVersion === prevDiagnosticVersion) { if (this._diagnosticVersion === prevDiagnosticVersion) {
return undefined; return undefined;
} }
@ -200,15 +200,11 @@ export class SourceFile {
} }
let diagList: Diagnostic[] = []; let diagList: Diagnostic[] = [];
diagList = diagList.concat( diagList = diagList.concat(this._parseDiagnostics, this._bindDiagnostics, this._checkerDiagnostics);
this._parseDiagnostics,
this._bindDiagnostics,
this._checkerDiagnostics);
// Filter the diagnostics based on "type: ignore" lines. // Filter the diagnostics based on "type: ignore" lines.
if (options.diagnosticSettings.enableTypeIgnoreComments) { if (options.diagnosticSettings.enableTypeIgnoreComments) {
const typeIgnoreLines = this._parseResults ? const typeIgnoreLines = this._parseResults ? this._parseResults.tokenizerOutput.typeIgnoreLines : {};
this._parseResults.tokenizerOutput.typeIgnoreLines : {};
if (Object.keys(typeIgnoreLines).length > 0) { if (Object.keys(typeIgnoreLines).length > 0) {
diagList = diagList.filter(d => { diagList = diagList.filter(d => {
for (let line = d.range.start.line; line <= d.range.end.line; line++) { for (let line = d.range.start.line; line <= d.range.end.line; line++) {
@ -223,19 +219,34 @@ export class SourceFile {
} }
if (options.diagnosticSettings.reportImportCycles !== 'none' && this._circularDependencies.length > 0) { if (options.diagnosticSettings.reportImportCycles !== 'none' && this._circularDependencies.length > 0) {
const category = options.diagnosticSettings.reportImportCycles === 'warning' ? const category =
DiagnosticCategory.Warning : DiagnosticCategory.Error; options.diagnosticSettings.reportImportCycles === 'warning'
? DiagnosticCategory.Warning
: DiagnosticCategory.Error;
this._circularDependencies.forEach(cirDep => { this._circularDependencies.forEach(cirDep => {
diagList.push(new Diagnostic(category, 'Cycle detected in import chain\n' + diagList.push(
cirDep.getPaths().map(path => ' ' + path).join('\n'), getEmptyRange())); new Diagnostic(
category,
'Cycle detected in import chain\n' +
cirDep
.getPaths()
.map(path => ' ' + path)
.join('\n'),
getEmptyRange()
)
);
}); });
} }
if (this._hitMaxImportDepth !== undefined) { if (this._hitMaxImportDepth !== undefined) {
diagList.push(new Diagnostic(DiagnosticCategory.Error, diagList.push(
`Import chain depth exceeded ${ this._hitMaxImportDepth }`, new Diagnostic(
getEmptyRange())); DiagnosticCategory.Error,
`Import chain depth exceeded ${this._hitMaxImportDepth}`,
getEmptyRange()
)
);
} }
if (this._isTypeshedStubFile) { if (this._isTypeshedStubFile) {
@ -245,8 +256,7 @@ export class SourceFile {
// Convert all the errors to warnings. // Convert all the errors to warnings.
diagList = diagList.map(diag => { diagList = diagList.map(diag => {
if (diag.category === DiagnosticCategory.Error) { if (diag.category === DiagnosticCategory.Error) {
return new Diagnostic(DiagnosticCategory.Warning, return new Diagnostic(DiagnosticCategory.Warning, diag.message, diag.range);
diag.message, diag.range);
} }
return diag; return diag;
}); });
@ -471,24 +481,30 @@ export class SourceFile {
// Resolve imports. // Resolve imports.
timingStats.resolveImportsTime.timeOperation(() => { timingStats.resolveImportsTime.timeOperation(() => {
[this._imports, this._builtinsImport, this._typingModulePath, this._collectionsModulePath] = [
this._resolveImports(importResolver, parseResults.importedModules, execEnvironment); this._imports,
this._builtinsImport,
this._typingModulePath,
this._collectionsModulePath
] = this._resolveImports(importResolver, parseResults.importedModules, execEnvironment);
this._parseDiagnostics = diagSink.fetchAndClear(); this._parseDiagnostics = diagSink.fetchAndClear();
}); });
// Is this file in a "strict" path? // Is this file in a "strict" path?
const useStrict = configOptions.strict.find( const useStrict =
strictFileSpec => strictFileSpec.regExp.test(this._filePath)) !== undefined; configOptions.strict.find(strictFileSpec => strictFileSpec.regExp.test(this._filePath)) !== undefined;
this._diagnosticSettings = CommentUtils.getFileLevelDirectives( this._diagnosticSettings = CommentUtils.getFileLevelDirectives(
this._parseResults.tokenizerOutput.tokens, configOptions.diagnosticSettings, this._parseResults.tokenizerOutput.tokens,
useStrict); configOptions.diagnosticSettings,
useStrict
);
} catch (e) { } catch (e) {
const message: string = (e.stack ? e.stack.toString() : undefined) || const message: string =
(e.stack ? e.stack.toString() : undefined) ||
(typeof e.message === 'string' ? e.message : undefined) || (typeof e.message === 'string' ? e.message : undefined) ||
JSON.stringify(e); JSON.stringify(e);
this._console.log( this._console.log(`An internal error occurred while parsing ${this.getFilePath()}: ` + message);
`An internal error occurred while parsing ${ this.getFilePath() }: ` + message);
// Create dummy parse results. // Create dummy parse results.
this._parseResults = { this._parseResults = {
@ -503,7 +519,7 @@ export class SourceFile {
typeIgnoreLines: {}, typeIgnoreLines: {},
predominantEndOfLineSequence: '\n', predominantEndOfLineSequence: '\n',
predominantTabSequence: ' ', predominantTabSequence: ' ',
predominantSingleQuoteCharacter: '\'' predominantSingleQuoteCharacter: "'"
}, },
containsWildcardImport: false containsWildcardImport: false
}; };
@ -525,41 +541,47 @@ export class SourceFile {
return true; return true;
} }
getDefinitionsForPosition(position: Position, getDefinitionsForPosition(position: Position, evaluator: TypeEvaluator): DocumentRange[] | undefined {
evaluator: TypeEvaluator): DocumentRange[] | undefined {
// If we have no completed analysis job, there's nothing to do. // If we have no completed analysis job, there's nothing to do.
if (!this._parseResults) { if (!this._parseResults) {
return undefined; return undefined;
} }
return DefinitionProvider.getDefinitionsForPosition( return DefinitionProvider.getDefinitionsForPosition(this._parseResults, position, evaluator);
this._parseResults, position, evaluator);
} }
getReferencesForPosition(position: Position, includeDeclaration: boolean, getReferencesForPosition(
evaluator: TypeEvaluator): ReferencesResult | undefined { position: Position,
includeDeclaration: boolean,
evaluator: TypeEvaluator
): ReferencesResult | undefined {
// If we have no completed analysis job, there's nothing to do. // If we have no completed analysis job, there's nothing to do.
if (!this._parseResults) { if (!this._parseResults) {
return undefined; return undefined;
} }
return ReferencesProvider.getReferencesForPosition(this._parseResults, return ReferencesProvider.getReferencesForPosition(
this._filePath, position, includeDeclaration, evaluator); this._parseResults,
this._filePath,
position,
includeDeclaration,
evaluator
);
} }
addReferences(referencesResult: ReferencesResult, includeDeclaration: boolean, addReferences(referencesResult: ReferencesResult, includeDeclaration: boolean, evaluator: TypeEvaluator): void {
evaluator: TypeEvaluator): void {
// If we have no completed analysis job, there's nothing to do. // If we have no completed analysis job, there's nothing to do.
if (!this._parseResults) { if (!this._parseResults) {
return; return;
} }
ReferencesProvider.addReferences( ReferencesProvider.addReferences(
this._parseResults, this._filePath, referencesResult, includeDeclaration, this._parseResults,
evaluator); this._filePath,
referencesResult,
includeDeclaration,
evaluator
);
} }
addHierarchicalSymbolsForDocument(symbolList: DocumentSymbol[], evaluator: TypeEvaluator) { addHierarchicalSymbolsForDocument(symbolList: DocumentSymbol[], evaluator: TypeEvaluator) {
@ -568,51 +590,49 @@ export class SourceFile {
return; return;
} }
DocumentSymbolProvider.addHierarchicalSymbolsForDocument(symbolList, DocumentSymbolProvider.addHierarchicalSymbolsForDocument(symbolList, this._parseResults, evaluator);
this._parseResults, evaluator);
} }
addSymbolsForDocument(symbolList: SymbolInformation[], evaluator: TypeEvaluator, addSymbolsForDocument(symbolList: SymbolInformation[], evaluator: TypeEvaluator, query?: string) {
query?: string) {
// If we have no completed analysis job, there's nothing to do. // If we have no completed analysis job, there's nothing to do.
if (!this._parseResults) { if (!this._parseResults) {
return; return;
} }
DocumentSymbolProvider.addSymbolsForDocument(symbolList, query, DocumentSymbolProvider.addSymbolsForDocument(symbolList, query, this._filePath, this._parseResults, evaluator);
this._filePath, this._parseResults, evaluator);
} }
getHoverForPosition(position: Position, getHoverForPosition(position: Position, evaluator: TypeEvaluator): HoverResults | undefined {
evaluator: TypeEvaluator): HoverResults | undefined {
// If this file hasn't been bound, no hover info is available. // If this file hasn't been bound, no hover info is available.
if (this._isBindingNeeded || !this._parseResults) { if (this._isBindingNeeded || !this._parseResults) {
return undefined; return undefined;
} }
return HoverProvider.getHoverForPosition( return HoverProvider.getHoverForPosition(this._parseResults, position, evaluator);
this._parseResults, position, evaluator);
} }
getSignatureHelpForPosition(position: Position, getSignatureHelpForPosition(
importLookup: ImportLookup, evaluator: TypeEvaluator): SignatureHelpResults | undefined { position: Position,
importLookup: ImportLookup,
evaluator: TypeEvaluator
): SignatureHelpResults | undefined {
// If we have no completed analysis job, there's nothing to do. // If we have no completed analysis job, there's nothing to do.
if (!this._parseResults) { if (!this._parseResults) {
return undefined; return undefined;
} }
return SignatureHelpProvider.getSignatureHelpForPosition( return SignatureHelpProvider.getSignatureHelpForPosition(this._parseResults, position, evaluator);
this._parseResults, position, evaluator);
} }
getCompletionsForPosition(position: Position, getCompletionsForPosition(
workspacePath: string, configOptions: ConfigOptions, importResolver: ImportResolver, position: Position,
importLookup: ImportLookup, evaluator: TypeEvaluator, workspacePath: string,
moduleSymbolsCallback: () => ModuleSymbolMap): CompletionList | undefined { configOptions: ConfigOptions,
importResolver: ImportResolver,
importLookup: ImportLookup,
evaluator: TypeEvaluator,
moduleSymbolsCallback: () => ModuleSymbolMap
): CompletionList | undefined {
// If we have no completed analysis job, there's nothing to do. // If we have no completed analysis job, there's nothing to do.
if (!this._parseResults) { if (!this._parseResults) {
return undefined; return undefined;
@ -625,28 +645,46 @@ export class SourceFile {
} }
const completionProvider = new CompletionProvider( const completionProvider = new CompletionProvider(
workspacePath, this._parseResults, this._fileContents, workspacePath,
importResolver, position, this._parseResults,
this._filePath, configOptions, importLookup, evaluator, this._fileContents,
moduleSymbolsCallback); importResolver,
position,
this._filePath,
configOptions,
importLookup,
evaluator,
moduleSymbolsCallback
);
return completionProvider.getCompletionsForPosition(); return completionProvider.getCompletionsForPosition();
} }
resolveCompletionItem(configOptions: ConfigOptions, importResolver: ImportResolver, resolveCompletionItem(
importLookup: ImportLookup, evaluator: TypeEvaluator, configOptions: ConfigOptions,
moduleSymbolsCallback: () => ModuleSymbolMap, completionItem: CompletionItem) { importResolver: ImportResolver,
importLookup: ImportLookup,
evaluator: TypeEvaluator,
moduleSymbolsCallback: () => ModuleSymbolMap,
completionItem: CompletionItem
) {
if (!this._parseResults || this._fileContents === undefined) { if (!this._parseResults || this._fileContents === undefined) {
return; return;
} }
const completionData = completionItem.data as CompletionItemData; const completionData = completionItem.data as CompletionItemData;
const completionProvider = new CompletionProvider( const completionProvider = new CompletionProvider(
completionData.workspacePath, this._parseResults, this._fileContents, completionData.workspacePath,
importResolver, completionData.position, this._parseResults,
this._filePath, configOptions, importLookup, evaluator, this._fileContents,
moduleSymbolsCallback); importResolver,
completionData.position,
this._filePath,
configOptions,
importLookup,
evaluator,
moduleSymbolsCallback
);
completionProvider.resolveCompletionItem(completionItem); completionProvider.resolveCompletionItem(completionItem);
} }
@ -681,8 +719,12 @@ export class SourceFile {
timingStats.bindTime.timeOperation(() => { timingStats.bindTime.timeOperation(() => {
this._cleanParseTreeIfRequired(); this._cleanParseTreeIfRequired();
const fileInfo = this._buildFileInfo(configOptions, this._parseResults!.text, const fileInfo = this._buildFileInfo(
importLookup, builtinsScope); configOptions,
this._parseResults!.text,
importLookup,
builtinsScope
);
AnalyzerNodeInfo.setFileInfo(this._parseResults!.parseTree, fileInfo); AnalyzerNodeInfo.setFileInfo(this._parseResults!.parseTree, fileInfo);
const binder = new Binder(fileInfo); const binder = new Binder(fileInfo);
@ -702,15 +744,16 @@ export class SourceFile {
this._moduleSymbolTable = moduleScope!.symbolTable; this._moduleSymbolTable = moduleScope!.symbolTable;
}); });
} catch (e) { } catch (e) {
const message: string = (e.stack ? e.stack.toString() : undefined) || const message: string =
(e.stack ? e.stack.toString() : undefined) ||
(typeof e.message === 'string' ? e.message : undefined) || (typeof e.message === 'string' ? e.message : undefined) ||
JSON.stringify(e); JSON.stringify(e);
this._console.log( this._console.log(
`An internal error occurred while performing name binding for ${ this.getFilePath() }: ` + message); `An internal error occurred while performing name binding for ${this.getFilePath()}: ` + message
);
const diagSink = new DiagnosticSink(); const diagSink = new DiagnosticSink();
diagSink.addError(`An internal error occurred while performing name binding`, diagSink.addError(`An internal error occurred while performing name binding`, getEmptyRange());
getEmptyRange());
this._bindDiagnostics = diagSink.fetchAndClear(); this._bindDiagnostics = diagSink.fetchAndClear();
} finally { } finally {
this._isBindingInProgress = false; this._isBindingInProgress = false;
@ -739,14 +782,15 @@ export class SourceFile {
this._checkerDiagnostics = fileInfo.diagnosticSink.fetchAndClear(); this._checkerDiagnostics = fileInfo.diagnosticSink.fetchAndClear();
}); });
} catch (e) { } catch (e) {
const message: string = (e.stack ? e.stack.toString() : undefined) || const message: string =
(e.stack ? e.stack.toString() : undefined) ||
(typeof e.message === 'string' ? e.message : undefined) || (typeof e.message === 'string' ? e.message : undefined) ||
JSON.stringify(e); JSON.stringify(e);
this._console.log( this._console.log(
`An internal error occurred while while performing type checking for ${ this.getFilePath() }: ` + message); `An internal error occurred while while performing type checking for ${this.getFilePath()}: ` + message
);
const diagSink = new DiagnosticSink(); const diagSink = new DiagnosticSink();
diagSink.addError(`An internal error occurred while performing type checking`, diagSink.addError(`An internal error occurred while performing type checking`, getEmptyRange());
getEmptyRange());
// Mark the file as complete so we don't get into an infinite loop. // Mark the file as complete so we don't get into an infinite loop.
this._isCheckingNeeded = false; this._isCheckingNeeded = false;
@ -760,9 +804,12 @@ export class SourceFile {
this._diagnosticVersion++; this._diagnosticVersion++;
} }
private _buildFileInfo(configOptions: ConfigOptions, fileContents: string, private _buildFileInfo(
importLookup: ImportLookup, builtinsScope?: Scope) { configOptions: ConfigOptions,
fileContents: string,
importLookup: ImportLookup,
builtinsScope?: Scope
) {
assert(this._parseResults !== undefined); assert(this._parseResults !== undefined);
const analysisDiagnostics = new TextRangeDiagnosticSink(this._parseResults!.tokenizerOutput.lines); const analysisDiagnostics = new TextRangeDiagnosticSink(this._parseResults!.tokenizerOutput.lines);
@ -796,45 +843,43 @@ export class SourceFile {
} }
} }
private _resolveImports(importResolver: ImportResolver, moduleImports: ModuleImport[], private _resolveImports(
execEnv: ExecutionEnvironment): [ImportResult[], ImportResult?, string?, string?] { importResolver: ImportResolver,
moduleImports: ModuleImport[],
execEnv: ExecutionEnvironment
): [ImportResult[], ImportResult?, string?, string?] {
const imports: ImportResult[] = []; const imports: ImportResult[] = [];
// Always include an implicit import of the builtins module. // Always include an implicit import of the builtins module.
let builtinsImportResult: ImportResult | undefined = importResolver.resolveImport( let builtinsImportResult: ImportResult | undefined = importResolver.resolveImport(this._filePath, execEnv, {
this._filePath, leadingDots: 0,
execEnv, nameParts: ['builtins'],
{ importedSymbols: undefined
leadingDots: 0, });
nameParts: ['builtins'],
importedSymbols: undefined
}
);
// Avoid importing builtins from the builtins.pyi file itself. // Avoid importing builtins from the builtins.pyi file itself.
if (builtinsImportResult.resolvedPaths.length === 0 || if (
builtinsImportResult.resolvedPaths[0] !== this.getFilePath()) { builtinsImportResult.resolvedPaths.length === 0 ||
builtinsImportResult.resolvedPaths[0] !== this.getFilePath()
) {
imports.push(builtinsImportResult); imports.push(builtinsImportResult);
} else { } else {
builtinsImportResult = undefined; builtinsImportResult = undefined;
} }
// Always include an implicit import of the typing module. // Always include an implicit import of the typing module.
const typingImportResult: ImportResult | undefined = importResolver.resolveImport( const typingImportResult: ImportResult | undefined = importResolver.resolveImport(this._filePath, execEnv, {
this._filePath, leadingDots: 0,
execEnv, nameParts: ['typing'],
{ importedSymbols: undefined
leadingDots: 0, });
nameParts: ['typing'],
importedSymbols: undefined
}
);
// Avoid importing typing from the typing.pyi file itself. // Avoid importing typing from the typing.pyi file itself.
let typingModulePath: string | undefined; let typingModulePath: string | undefined;
if (typingImportResult.resolvedPaths.length === 0 || if (
typingImportResult.resolvedPaths[0] !== this.getFilePath()) { typingImportResult.resolvedPaths.length === 0 ||
typingImportResult.resolvedPaths[0] !== this.getFilePath()
) {
imports.push(typingImportResult); imports.push(typingImportResult);
typingModulePath = typingImportResult.resolvedPaths[0]; typingModulePath = typingImportResult.resolvedPaths[0];
} }
@ -842,15 +887,11 @@ export class SourceFile {
let collectionsModulePath: string | undefined; let collectionsModulePath: string | undefined;
for (const moduleImport of moduleImports) { for (const moduleImport of moduleImports) {
const importResult = importResolver.resolveImport( const importResult = importResolver.resolveImport(this._filePath, execEnv, {
this._filePath, leadingDots: moduleImport.leadingDots,
execEnv, nameParts: moduleImport.nameParts,
{ importedSymbols: moduleImport.importedSymbols
leadingDots: moduleImport.leadingDots, });
nameParts: moduleImport.nameParts,
importedSymbols: moduleImport.importedSymbols
}
);
// If the file imports the stdlib 'collections' module, stash // If the file imports the stdlib 'collections' module, stash
// away its file path. The type analyzer may need this to // away its file path. The type analyzer may need this to

137
server/src/analyzer/staticExpressions.ts
View File

@ -1,12 +1,12 @@
/* /*
* staticExpressions.ts * staticExpressions.ts
* Copyright (c) Microsoft Corporation. * Copyright (c) Microsoft Corporation.
* Licensed under the MIT license. * Licensed under the MIT license.
* Author: Eric Traut * Author: Eric Traut
* *
* Collection of static methods that operate on expressions * Collection of static methods that operate on expressions
* (parse node trees). * (parse node trees).
*/ */
import { ExecutionEnvironment } from '../common/configOptions'; import { ExecutionEnvironment } from '../common/configOptions';
import { ExpressionNode, NumberNode, ParseNodeType, TupleNode } from '../parser/parseNodes'; import { ExpressionNode, NumberNode, ParseNodeType, TupleNode } from '../parser/parseNodes';
@ -14,9 +14,7 @@ import { KeywordType, OperatorType } from '../parser/tokenizerTypes';
// Returns undefined if the expression cannot be evaluated // Returns undefined if the expression cannot be evaluated
// statically as a bool value or true/false if it can. // statically as a bool value or true/false if it can.
export function evaluateStaticBoolExpression(node: ExpressionNode, export function evaluateStaticBoolExpression(node: ExpressionNode, execEnv: ExecutionEnvironment): boolean | undefined {
execEnv: ExecutionEnvironment): boolean | undefined {
if (node.nodeType === ParseNodeType.UnaryOperation) { if (node.nodeType === ParseNodeType.UnaryOperation) {
if (node.operator === OperatorType.Or || node.operator === OperatorType.And) { if (node.operator === OperatorType.Or || node.operator === OperatorType.And) {
const value = evaluateStaticBoolLikeExpression(node.expression, execEnv); const value = evaluateStaticBoolLikeExpression(node.expression, execEnv);
@ -41,40 +39,42 @@ export function evaluateStaticBoolExpression(node: ExpressionNode,
} }
} }
if (_isSysVersionInfoExpression(node.leftExpression) && if (_isSysVersionInfoExpression(node.leftExpression) && node.rightExpression.nodeType === ParseNodeType.Tuple) {
node.rightExpression.nodeType === ParseNodeType.Tuple) {
// Handle the special case of "sys.version_info >= (3, x)" // Handle the special case of "sys.version_info >= (3, x)"
const comparisonVersion = _convertTupleToVersion(node.rightExpression); const comparisonVersion = _convertTupleToVersion(node.rightExpression);
return _evaluateNumericBinaryOperation(node.operator, return _evaluateNumericBinaryOperation(node.operator, execEnv.pythonVersion, comparisonVersion);
execEnv.pythonVersion, comparisonVersion); } else if (
node.leftExpression.nodeType === ParseNodeType.Index &&
} else if (node.leftExpression.nodeType === ParseNodeType.Index && _isSysVersionInfoExpression(node.leftExpression.baseExpression) &&
_isSysVersionInfoExpression(node.leftExpression.baseExpression) && node.leftExpression.items.items.length === 1 &&
node.leftExpression.items.items.length === 1 && node.leftExpression.items.items[0].nodeType === ParseNodeType.Number &&
node.leftExpression.items.items[0].nodeType === ParseNodeType.Number && !node.leftExpression.items.items[0].isImaginary &&
!node.leftExpression.items.items[0].isImaginary && node.leftExpression.items.items[0].value === 0 &&
node.leftExpression.items.items[0].value === 0 && node.rightExpression.nodeType === ParseNodeType.Number
node.rightExpression.nodeType === ParseNodeType.Number) { ) {
// Handle the special case of "sys.version_info[0] >= X" // Handle the special case of "sys.version_info[0] >= X"
return _evaluateNumericBinaryOperation(node.operator, return _evaluateNumericBinaryOperation(
Math.floor(execEnv.pythonVersion / 256), node.rightExpression.value); node.operator,
} else if (_isSysPlatformInfoExpression(node.leftExpression) && Math.floor(execEnv.pythonVersion / 256),
node.rightExpression.nodeType === ParseNodeType.StringList) { node.rightExpression.value
);
} else if (
_isSysPlatformInfoExpression(node.leftExpression) &&
node.rightExpression.nodeType === ParseNodeType.StringList
) {
// Handle the special case of "sys.platform != 'X'" // Handle the special case of "sys.platform != 'X'"
const comparisonPlatform = node.rightExpression.strings.map(s => s.value).join(''); const comparisonPlatform = node.rightExpression.strings.map(s => s.value).join('');
const expectedPlatformName = _getExpectedPlatformNameFromPlatform(execEnv); const expectedPlatformName = _getExpectedPlatformNameFromPlatform(execEnv);
return _evaluateStringBinaryOperation(node.operator, return _evaluateStringBinaryOperation(node.operator, expectedPlatformName || '', comparisonPlatform);
expectedPlatformName || '', comparisonPlatform); } else if (
} else if (_isOsNameInfoExpression(node.leftExpression) && _isOsNameInfoExpression(node.leftExpression) &&
node.rightExpression.nodeType === ParseNodeType.StringList) { node.rightExpression.nodeType === ParseNodeType.StringList
) {
// Handle the special case of "os.name == 'X'" // Handle the special case of "os.name == 'X'"
const comparisonOsName = node.rightExpression.strings.map(s => s.value).join(''); const comparisonOsName = node.rightExpression.strings.map(s => s.value).join('');
const expectedOsName = _getExpectedOsNameFromPlatform(execEnv); const expectedOsName = _getExpectedOsNameFromPlatform(execEnv);
if (expectedOsName !== undefined) { if (expectedOsName !== undefined) {
return _evaluateStringBinaryOperation(node.operator, return _evaluateStringBinaryOperation(node.operator, expectedOsName, comparisonOsName);
expectedOsName, comparisonOsName);
} }
} }
} else if (node.nodeType === ParseNodeType.Constant) { } else if (node.nodeType === ParseNodeType.Constant) {
@ -87,11 +87,12 @@ export function evaluateStaticBoolExpression(node: ExpressionNode,
if (node.value === 'TYPE_CHECKING') { if (node.value === 'TYPE_CHECKING') {
return true; return true;
} }
} else if (node.nodeType === ParseNodeType.MemberAccess && } else if (
node.memberName.value === 'TYPE_CHECKING' && node.nodeType === ParseNodeType.MemberAccess &&
node.leftExpression.nodeType === ParseNodeType.Name && node.memberName.value === 'TYPE_CHECKING' &&
node.leftExpression.value === 'typing') { node.leftExpression.nodeType === ParseNodeType.Name &&
node.leftExpression.value === 'typing'
) {
return true; return true;
} }
@ -101,9 +102,10 @@ export function evaluateStaticBoolExpression(node: ExpressionNode,
// Similar to evaluateStaticBoolExpression except that it handles // Similar to evaluateStaticBoolExpression except that it handles
// other non-bool values that are statically falsy or truthy // other non-bool values that are statically falsy or truthy
// (like "None"). // (like "None").
export function evaluateStaticBoolLikeExpression(node: ExpressionNode, export function evaluateStaticBoolLikeExpression(
execEnv: ExecutionEnvironment): boolean | undefined { node: ExpressionNode,
execEnv: ExecutionEnvironment
): boolean | undefined {
if (node.nodeType === ParseNodeType.Constant) { if (node.nodeType === ParseNodeType.Constant) {
if (node.constType === KeywordType.None) { if (node.constType === KeywordType.None) {
return false; return false;
@ -116,11 +118,12 @@ export function evaluateStaticBoolLikeExpression(node: ExpressionNode,
function _convertTupleToVersion(node: TupleNode): number | undefined { function _convertTupleToVersion(node: TupleNode): number | undefined {
let comparisonVersion: number | undefined; let comparisonVersion: number | undefined;
if (node.expressions.length === 2) { if (node.expressions.length === 2) {
if (node.expressions[0].nodeType === ParseNodeType.Number && if (
!node.expressions[0].isImaginary && node.expressions[0].nodeType === ParseNodeType.Number &&
node.expressions[1].nodeType === ParseNodeType.Number && !node.expressions[0].isImaginary &&
!node.expressions[1].isImaginary) { node.expressions[1].nodeType === ParseNodeType.Number &&
!node.expressions[1].isImaginary
) {
const majorVersion = node.expressions[0]; const majorVersion = node.expressions[0];
const minorVersion = node.expressions[1]; const minorVersion = node.expressions[1];
comparisonVersion = majorVersion.value * 256 + minorVersion.value; comparisonVersion = majorVersion.value * 256 + minorVersion.value;
@ -133,8 +136,11 @@ function _convertTupleToVersion(node: TupleNode): number | undefined {
return comparisonVersion; return comparisonVersion;
} }
function _evaluateNumericBinaryOperation(operatorType: OperatorType, leftValue: number | undefined, function _evaluateNumericBinaryOperation(
rightValue: number | undefined): any | undefined { operatorType: OperatorType,
leftValue: number | undefined,
rightValue: number | undefined
): any | undefined {
if (leftValue !== undefined && rightValue !== undefined) { if (leftValue !== undefined && rightValue !== undefined) {
if (operatorType === OperatorType.LessThan) { if (operatorType === OperatorType.LessThan) {
return leftValue < rightValue; return leftValue < rightValue;
@ -154,8 +160,11 @@ function _evaluateNumericBinaryOperation(operatorType: OperatorType, leftValue:
return undefined; return undefined;
} }
function _evaluateStringBinaryOperation(operatorType: OperatorType, function _evaluateStringBinaryOperation(
leftValue: string | undefined, rightValue: string | undefined): any | undefined { operatorType: OperatorType,
leftValue: string | undefined,
rightValue: string | undefined
): any | undefined {
if (leftValue !== undefined && rightValue !== undefined) { if (leftValue !== undefined && rightValue !== undefined) {
if (operatorType === OperatorType.Equals) { if (operatorType === OperatorType.Equals) {
return leftValue === rightValue; return leftValue === rightValue;
@ -169,9 +178,11 @@ function _evaluateStringBinaryOperation(operatorType: OperatorType,
function _isSysVersionInfoExpression(node: ExpressionNode): boolean { function _isSysVersionInfoExpression(node: ExpressionNode): boolean {
if (node.nodeType === ParseNodeType.MemberAccess) { if (node.nodeType === ParseNodeType.MemberAccess) {
if (node.leftExpression.nodeType === ParseNodeType.Name && if (
node.leftExpression.value === 'sys' && node.leftExpression.nodeType === ParseNodeType.Name &&
node.memberName.value === 'version_info') { node.leftExpression.value === 'sys' &&
node.memberName.value === 'version_info'
) {
return true; return true;
} }
} }
@ -181,9 +192,11 @@ function _isSysVersionInfoExpression(node: ExpressionNode): boolean {
function _isSysPlatformInfoExpression(node: ExpressionNode): boolean { function _isSysPlatformInfoExpression(node: ExpressionNode): boolean {
if (node.nodeType === ParseNodeType.MemberAccess) { if (node.nodeType === ParseNodeType.MemberAccess) {
if (node.leftExpression.nodeType === ParseNodeType.Name && if (
node.leftExpression.value === 'sys' && node.leftExpression.nodeType === ParseNodeType.Name &&
node.memberName.value === 'platform') { node.leftExpression.value === 'sys' &&
node.memberName.value === 'platform'
) {
return true; return true;
} }
} }
@ -193,9 +206,11 @@ function _isSysPlatformInfoExpression(node: ExpressionNode): boolean {
function _isOsNameInfoExpression(node: ExpressionNode): boolean { function _isOsNameInfoExpression(node: ExpressionNode): boolean {
if (node.nodeType === ParseNodeType.MemberAccess) { if (node.nodeType === ParseNodeType.MemberAccess) {
if (node.leftExpression.nodeType === ParseNodeType.Name && if (
node.leftExpression.value === 'os' && node.leftExpression.nodeType === ParseNodeType.Name &&
node.memberName.value === 'name') { node.leftExpression.value === 'os' &&
node.memberName.value === 'name'
) {
return true; return true;
} }
} }

31
server/src/analyzer/symbol.ts
View File

@ -1,13 +1,13 @@
/* /*
* symbol.ts * symbol.ts
* Copyright (c) Microsoft Corporation. * Copyright (c) Microsoft Corporation.
* Licensed under the MIT license. * Licensed under the MIT license.
* Author: Eric Traut * Author: Eric Traut
* *
* Represents an association between a name and the type * Represents an association between a name and the type
* (or multiple types) that the symbol is associated with * (or multiple types) that the symbol is associated with
* in the program. * in the program.
*/ */
import { Declaration, DeclarationType } from './declaration'; import { Declaration, DeclarationType } from './declaration';
import { areDeclarationsSame, hasTypeForDeclaration } from './declarationUtils'; import { areDeclarationsSame, hasTypeForDeclaration } from './declarationUtils';
@ -129,8 +129,7 @@ export class Symbol {
// See if this node was already identified as a declaration. If so, // See if this node was already identified as a declaration. If so,
// replace it. Otherwise, add it as a new declaration to the end of // replace it. Otherwise, add it as a new declaration to the end of
// the list. // the list.
const declIndex = this._declarations.findIndex( const declIndex = this._declarations.findIndex(decl => areDeclarationsSame(decl, declaration));
decl => areDeclarationsSame(decl, declaration));
if (declIndex < 0) { if (declIndex < 0) {
this._declarations.push(declaration); this._declarations.push(declaration);
} else { } else {
@ -139,9 +138,7 @@ export class Symbol {
const curDecl = this._declarations[declIndex]; const curDecl = this._declarations[declIndex];
if (hasTypeForDeclaration(declaration)) { if (hasTypeForDeclaration(declaration)) {
this._declarations[declIndex] = declaration; this._declarations[declIndex] = declaration;
if (curDecl.type === DeclarationType.Variable && if (curDecl.type === DeclarationType.Variable && declaration.type === DeclarationType.Variable) {
declaration.type === DeclarationType.Variable) {
if (!declaration.inferredTypeSource && curDecl.inferredTypeSource) { if (!declaration.inferredTypeSource && curDecl.inferredTypeSource) {
declaration.inferredTypeSource = curDecl.inferredTypeSource; declaration.inferredTypeSource = curDecl.inferredTypeSource;
} }
@ -180,13 +177,11 @@ export class Symbol {
return true; return true;
} }
return this.getDeclarations().some( return this.getDeclarations().some(decl => hasTypeForDeclaration(decl));
decl => hasTypeForDeclaration(decl));
} }
getTypedDeclarations() { getTypedDeclarations() {
return this.getDeclarations().filter( return this.getDeclarations().filter(decl => hasTypeForDeclaration(decl));
decl => hasTypeForDeclaration(decl));
} }
getSynthesizedType() { getSynthesizedType() {

26
server/src/analyzer/symbolNameUtils.ts
View File

@ -1,27 +1,23 @@
/* /*
* symbolNameUtils.ts * symbolNameUtils.ts
* Copyright (c) Microsoft Corporation. * Copyright (c) Microsoft Corporation.
* Licensed under the MIT license. * Licensed under the MIT license.
* Author: Eric Traut * Author: Eric Traut
* *
* Static methods that apply to symbols or symbol names. * Static methods that apply to symbols or symbol names.
*/ */
const _constantRegEx = /^[A-Z0-9_]+$/; const _constantRegEx = /^[A-Z0-9_]+$/;
const _underscoreOnlyRegEx = /^[_]+$/; const _underscoreOnlyRegEx = /^[_]+$/;
// Private symbol names start with a double underscore. // Private symbol names start with a double underscore.
export function isPrivateName(name: string) { export function isPrivateName(name: string) {
return name.length > 2 && return name.length > 2 && name.startsWith('__') && !name.endsWith('__');
name.startsWith('__') &&
!name.endsWith('__');
} }
// Protected symbol names start with a single underscore. // Protected symbol names start with a single underscore.
export function isProtectedName(name: string) { export function isProtectedName(name: string) {
return name.length > 1 && return name.length > 1 && name.startsWith('_') && !name.startsWith('__');
name.startsWith('_') &&
!name.startsWith('__');
} }
export function isPrivateOrProtectedName(name: string) { export function isPrivateOrProtectedName(name: string) {
@ -30,9 +26,7 @@ export function isPrivateOrProtectedName(name: string) {
// "Dunder" names start and end with two underscores. // "Dunder" names start and end with two underscores.
export function isDunderName(name: string) { export function isDunderName(name: string) {
return name.length > 4 && return name.length > 4 && name.startsWith('__') && name.endsWith('__');
name.startsWith('__') &&
name.endsWith('__');
} }
// Constants are all-caps with possible numbers and underscores. // Constants are all-caps with possible numbers and underscores.

14
server/src/analyzer/symbolUtils.ts
View File

@ -1,11 +1,11 @@
/* /*
* symbolUtils.ts * symbolUtils.ts
* Copyright (c) Microsoft Corporation. * Copyright (c) Microsoft Corporation.
* Licensed under the MIT license. * Licensed under the MIT license.
* Author: Eric Traut * Author: Eric Traut
* *
* Collection of functions that operate on Symbol objects. * Collection of functions that operate on Symbol objects.
*/ */
import { Declaration, DeclarationType } from './declaration'; import { Declaration, DeclarationType } from './declaration';
import { isFinalVariableDeclaration } from './declarationUtils'; import { isFinalVariableDeclaration } from './declarationUtils';

16
server/src/analyzer/testWalker.ts
View File

@ -1,8 +1,8 @@
/* /*
* testWalker.ts * testWalker.ts
* *
* Walks a parse tree to validate internal consistency and completeness. * Walks a parse tree to validate internal consistency and completeness.
*/ */
import { ParseTreeWalker } from '../analyzer/parseTreeWalker'; import { ParseTreeWalker } from '../analyzer/parseTreeWalker';
import { fail } from '../common/debug'; import { fail } from '../common/debug';
@ -27,8 +27,9 @@ export class TestWalker extends ParseTreeWalker {
children.forEach(child => { children.forEach(child => {
if (child) { if (child) {
if (child.parent !== node) { if (child.parent !== node) {
fail(`Child node ${ child.nodeType } does not ` + fail(
`contain a reference to its parent ${ node.nodeType }`); `Child node ${child.nodeType} does not ` + `contain a reference to its parent ${node.nodeType}`
);
} }
} }
}); });
@ -62,8 +63,7 @@ export class TestWalker extends ParseTreeWalker {
if (!skipCheck) { if (!skipCheck) {
// Make sure the child is contained within the parent. // Make sure the child is contained within the parent.
if (child.start < node.start || TextRange.getEnd(child) > TextRange.getEnd(node)) { if (child.start < node.start || TextRange.getEnd(child) > TextRange.getEnd(node)) {
fail(`Child node ${ child.nodeType } is not ` + fail(`Child node ${child.nodeType} is not ` + `contained within its parent ${node.nodeType}`);
`contained within its parent ${ node.nodeType }`);
} }
if (prevNode) { if (prevNode) {
// Make sure the child is after the previous child. // Make sure the child is after the previous child.

3460
server/src/analyzer/typeEvaluator.ts
View File

File diff suppressed because it is too large Load Diff

176
server/src/analyzer/typeStubWriter.ts
View File

@ -1,19 +1,38 @@
/* /*
* typeStubWriter.ts * typeStubWriter.ts
* Copyright (c) Microsoft Corporation. * Copyright (c) Microsoft Corporation.
* Licensed under the MIT license. * Licensed under the MIT license.
* Author: Eric Traut * Author: Eric Traut
* *
* Logic to emit a type stub file for a corresponding parsed * Logic to emit a type stub file for a corresponding parsed
* and analyzed python source file. * and analyzed python source file.
*/ */
import { import {
ArgumentCategory, ArgumentNode, AssignmentNode, AugmentedAssignmentNode, ArgumentCategory,
ClassNode, DecoratorNode, ExpressionNode, ForNode, FunctionNode, IfNode, ArgumentNode,
ImportFromNode, ImportNode, ModuleNameNode, NameNode, ParameterCategory, ParameterNode, AssignmentNode,
ParseNode, ParseNodeType, StatementListNode, StringNode, TryNode, AugmentedAssignmentNode,
TypeAnnotationNode, WhileNode, WithNode ClassNode,
DecoratorNode,
ExpressionNode,
ForNode,
FunctionNode,
IfNode,
ImportFromNode,
ImportNode,
ModuleNameNode,
NameNode,
ParameterCategory,
ParameterNode,
ParseNode,
ParseNodeType,
StatementListNode,
StringNode,
TryNode,
TypeAnnotationNode,
WhileNode,
WithNode
} from '../parser/parseNodes'; } from '../parser/parseNodes';
import * as AnalyzerNodeInfo from './analyzerNodeInfo'; import * as AnalyzerNodeInfo from './analyzerNodeInfo';
import * as ParseTreeUtils from './parseTreeUtils'; import * as ParseTreeUtils from './parseTreeUtils';
@ -27,15 +46,13 @@ import { ClassType, isNoneOrNever, TypeCategory } from './types';
import * as TypeUtils from './typeUtils'; import * as TypeUtils from './typeUtils';
class TrackedImport { class TrackedImport {
constructor(public importName: string) { } constructor(public importName: string) {}
isAccessed = false; isAccessed = false;
} }
class TrackedImportAs extends TrackedImport { class TrackedImportAs extends TrackedImport {
constructor(importName: string, public alias: string | undefined, constructor(importName: string, public alias: string | undefined, public symbol: Symbol) {
public symbol: Symbol) {
super(importName); super(importName);
} }
} }
@ -54,9 +71,7 @@ class TrackedImportFrom extends TrackedImport {
super(importName); super(importName);
} }
addSymbol(symbol: Symbol | undefined, name: string, addSymbol(symbol: Symbol | undefined, name: string, alias: string | undefined, isAccessed = false) {
alias: string | undefined, isAccessed = false) {
if (!this.symbols.find(s => s.name === name)) { if (!this.symbols.find(s => s.name === name)) {
this.symbols.push({ this.symbols.push({
symbol, symbol,
@ -69,10 +84,7 @@ class TrackedImportFrom extends TrackedImport {
} }
class ImportSymbolWalker extends ParseTreeWalker { class ImportSymbolWalker extends ParseTreeWalker {
constructor( constructor(private _accessedImportedSymbols: Map<string, boolean>, private _treatStringsAsSymbols: boolean) {
private _accessedImportedSymbols: Map<string, boolean>,
private _treatStringsAsSymbols: boolean) {
super(); super();
} }
@ -115,8 +127,7 @@ export class TypeStubWriter extends ParseTreeWalker {
private _trackedImportFrom = new Map<string, TrackedImportFrom>(); private _trackedImportFrom = new Map<string, TrackedImportFrom>();
private _accessedImportedSymbols = new Map<string, boolean>(); private _accessedImportedSymbols = new Map<string, boolean>();
constructor(private _typingsPath: string, private _sourceFile: SourceFile, constructor(private _typingsPath: string, private _sourceFile: SourceFile, private _evaluator: TypeEvaluator) {
private _evaluator: TypeEvaluator) {
super(); super();
// As a heuristic, we'll include all of the import statements // As a heuristic, we'll include all of the import statements
@ -149,16 +160,18 @@ export class TypeStubWriter extends ParseTreeWalker {
this._emittedSuite = true; this._emittedSuite = true;
this._emitDocString = true; this._emitDocString = true;
this._emitDecorators(node.decorators); this._emitDecorators(node.decorators);
let line = `class ${ className }`; let line = `class ${className}`;
if (node.arguments.length > 0) { if (node.arguments.length > 0) {
line += `(${ node.arguments.map(arg => { line += `(${node.arguments
let argString = ''; .map(arg => {
if (arg.name) { let argString = '';
argString = arg.name.value + '='; if (arg.name) {
} argString = arg.name.value + '=';
argString += this._printExpression(arg.valueExpression); }
return argString; argString += this._printExpression(arg.valueExpression);
}).join(', ') })`; return argString;
})
.join(', ')})`;
} }
line += ':'; line += ':';
this._emitLine(line); this._emitLine(line);
@ -184,8 +197,8 @@ export class TypeStubWriter extends ParseTreeWalker {
this._emitDocString = true; this._emitDocString = true;
this._emitDecorators(node.decorators); this._emitDecorators(node.decorators);
let line = node.isAsync ? 'async ' : ''; let line = node.isAsync ? 'async ' : '';
line += `def ${ functionName }`; line += `def ${functionName}`;
line += `(${ node.parameters.map(param => this._printParameter(param)).join(', ') })`; line += `(${node.parameters.map(param => this._printParameter(param)).join(', ')})`;
if (node.returnTypeAnnotation) { if (node.returnTypeAnnotation) {
line += ' -> ' + this._printExpression(node.returnTypeAnnotation, true); line += ' -> ' + this._printExpression(node.returnTypeAnnotation, true);
@ -357,14 +370,18 @@ export class TypeStubWriter extends ParseTreeWalker {
node.list.forEach(imp => { node.list.forEach(imp => {
const moduleName = this._printModuleName(imp.module); const moduleName = this._printModuleName(imp.module);
if (!this._trackedImportAs.has(moduleName)) { if (!this._trackedImportAs.has(moduleName)) {
const symbolName = imp.alias ? imp.alias.value : const symbolName = imp.alias
(imp.module.nameParts.length > 0 ? ? imp.alias.value
imp.module.nameParts[0].value : ''); : imp.module.nameParts.length > 0
? imp.module.nameParts[0].value
: '';
const symbolInfo = currentScope.lookUpSymbolRecursive(symbolName); const symbolInfo = currentScope.lookUpSymbolRecursive(symbolName);
if (symbolInfo) { if (symbolInfo) {
const trackedImportAs = new TrackedImportAs(moduleName, const trackedImportAs = new TrackedImportAs(
moduleName,
imp.alias ? imp.alias.value : undefined, imp.alias ? imp.alias.value : undefined,
symbolInfo.symbol); symbolInfo.symbol
);
this._trackedImportAs.set(moduleName, trackedImportAs); this._trackedImportAs.set(moduleName, trackedImportAs);
} }
} }
@ -385,18 +402,20 @@ export class TypeStubWriter extends ParseTreeWalker {
const moduleName = this._printModuleName(node.module); const moduleName = this._printModuleName(node.module);
let trackedImportFrom = this._trackedImportFrom.get(moduleName); let trackedImportFrom = this._trackedImportFrom.get(moduleName);
if (!trackedImportFrom) { if (!trackedImportFrom) {
trackedImportFrom = new TrackedImportFrom(moduleName, trackedImportFrom = new TrackedImportFrom(moduleName, node.isWildcardImport, node);
node.isWildcardImport, node);
this._trackedImportFrom.set(moduleName, trackedImportFrom); this._trackedImportFrom.set(moduleName, trackedImportFrom);
} }
node.imports.forEach(imp => { node.imports.forEach(imp => {
const symbolName = imp.alias ? const symbolName = imp.alias ? imp.alias.value : imp.name.value;
imp.alias.value : imp.name.value;
const symbolInfo = currentScope.lookUpSymbolRecursive(symbolName); const symbolInfo = currentScope.lookUpSymbolRecursive(symbolName);
if (symbolInfo) { if (symbolInfo) {
trackedImportFrom!.addSymbol(symbolInfo.symbol, imp.name.value, trackedImportFrom!.addSymbol(
imp.alias ? imp.alias.value : undefined, false); symbolInfo.symbol,
imp.name.value,
imp.alias ? imp.alias.value : undefined,
false
);
} }
}); });
} }
@ -445,18 +464,22 @@ export class TypeStubWriter extends ParseTreeWalker {
decorators.forEach(decorator => { decorators.forEach(decorator => {
let line = '@' + this._printExpression(decorator.leftExpression); let line = '@' + this._printExpression(decorator.leftExpression);
if (decorator.arguments) { if (decorator.arguments) {
line += `(${ decorator.arguments.map( line += `(${decorator.arguments.map(arg => this._printArgument(arg)).join(', ')})`;
arg => this._printArgument(arg)).join(', ') })`;
} }
this._emitLine(line); this._emitLine(line);
}); });
} }
private _printHeaderDocString() { private _printHeaderDocString() {
return '"""' + this._lineEnd + return (
'This type stub file was generated by pyright.' + this._lineEnd + '"""' +
'"""' + this._lineEnd + this._lineEnd +
this._lineEnd; 'This type stub file was generated by pyright.' +
this._lineEnd +
'"""' +
this._lineEnd +
this._lineEnd
);
// this._emitLine(''); // this._emitLine('');
// this._emitLine('from typing import Any, Optional'); // this._emitLine('from typing import Any, Optional');
// this._emitLine('from typing import Any, List, Dict, Optional, Tuple, Type, Union'); // this._emitLine('from typing import Any, List, Dict, Optional, Tuple, Type, Union');
@ -555,17 +578,14 @@ export class TypeStubWriter extends ParseTreeWalker {
return line + this._printExpression(node.valueExpression); return line + this._printExpression(node.valueExpression);
} }
private _printExpression(node: ExpressionNode, isType = false, private _printExpression(node: ExpressionNode, isType = false, treatStringsAsSymbols = false): string {
treatStringsAsSymbols = false): string { const importSymbolWalker = new ImportSymbolWalker(this._accessedImportedSymbols, treatStringsAsSymbols);
const importSymbolWalker = new ImportSymbolWalker(
this._accessedImportedSymbols,
treatStringsAsSymbols);
importSymbolWalker.analyze(node); importSymbolWalker.analyze(node);
return ParseTreeUtils.printExpression(node, return ParseTreeUtils.printExpression(
isType ? ParseTreeUtils.PrintExpressionFlags.ForwardDeclarations : node,
ParseTreeUtils.PrintExpressionFlags.None); isType ? ParseTreeUtils.PrintExpressionFlags.ForwardDeclarations : ParseTreeUtils.PrintExpressionFlags.None
);
} }
private _printTrackedImports() { private _printTrackedImports() {
@ -579,9 +599,9 @@ export class TypeStubWriter extends ParseTreeWalker {
} }
if (imp.isAccessed || this._includeAllImports) { if (imp.isAccessed || this._includeAllImports) {
importStr += `import ${ imp.importName }`; importStr += `import ${imp.importName}`;
if (imp.alias) { if (imp.alias) {
importStr += ` as ${ imp.alias }`; importStr += ` as ${imp.alias}`;
} }
importStr += this._lineEnd; importStr += this._lineEnd;
lineEmitted = true; lineEmitted = true;
@ -597,13 +617,13 @@ export class TypeStubWriter extends ParseTreeWalker {
}); });
if (imp.isWildcardImport) { if (imp.isWildcardImport) {
importStr += `from ${ imp.importName } import *` + this._lineEnd; importStr += `from ${imp.importName} import *` + this._lineEnd;
lineEmitted = true; lineEmitted = true;
} }
const sortedSymbols = imp.symbols. const sortedSymbols = imp.symbols
filter(s => s.isAccessed || this._includeAllImports). .filter(s => s.isAccessed || this._includeAllImports)
sort((a, b) => { .sort((a, b) => {
if (a.name < b.name) { if (a.name < b.name) {
return -1; return -1;
} else if (a.name > b.name) { } else if (a.name > b.name) {
@ -613,15 +633,17 @@ export class TypeStubWriter extends ParseTreeWalker {
}); });
if (sortedSymbols.length > 0) { if (sortedSymbols.length > 0) {
importStr += `from ${ imp.importName } import `; importStr += `from ${imp.importName} import `;
importStr += sortedSymbols.map(symbol => { importStr += sortedSymbols
let symStr = symbol.name; .map(symbol => {
if (symbol.alias) { let symStr = symbol.name;
symStr += ' as ' + symbol.alias; if (symbol.alias) {
} symStr += ' as ' + symbol.alias;
return symStr; }
}).join(', '); return symStr;
})
.join(', ');
importStr += this._lineEnd; importStr += this._lineEnd;
lineEmitted = true; lineEmitted = true;

284
server/src/analyzer/typeUtils.ts
View File

@ -1,21 +1,36 @@
/* /*
* typeUtils.ts * typeUtils.ts
* Copyright (c) Microsoft Corporation. * Copyright (c) Microsoft Corporation.
* Licensed under the MIT license. * Licensed under the MIT license.
* Author: Eric Traut * Author: Eric Traut
* *
* Collection of functions that operate on Type objects. * Collection of functions that operate on Type objects.
*/ */
import { ParameterCategory } from '../parser/parseNodes'; import { ParameterCategory } from '../parser/parseNodes';
import { ImportLookup } from './analyzerFileInfo'; import { ImportLookup } from './analyzerFileInfo';
import { DeclarationType } from './declaration'; import { DeclarationType } from './declaration';
import { Symbol, SymbolFlags, SymbolTable } from './symbol'; import { Symbol, SymbolFlags, SymbolTable } from './symbol';
import { isTypedDictMemberAccessedThroughIndex } from './symbolUtils'; import { isTypedDictMemberAccessedThroughIndex } from './symbolUtils';
import { AnyType, ClassType, combineTypes, FunctionType, isAnyOrUnknown, isNoneOrNever, import {
isTypeSame, maxTypeRecursionCount, ModuleType, NeverType, ObjectType, AnyType,
OverloadedFunctionType, SpecializedFunctionTypes, Type, TypeCategory, ClassType,
TypeVarType, UnknownType } from './types'; combineTypes,
FunctionType,
isAnyOrUnknown,
isNoneOrNever,
isTypeSame,
maxTypeRecursionCount,
ModuleType,
NeverType,
ObjectType,
OverloadedFunctionType,
SpecializedFunctionTypes,
Type,
TypeCategory,
TypeVarType,
UnknownType
} from './types';
import { TypeVarMap } from './typeVarMap'; import { TypeVarMap } from './typeVarMap';
export interface ClassMember { export interface ClassMember {
@ -89,7 +104,7 @@ export function isOptionalType(type: Type): boolean {
// Calls a callback for each subtype and combines the results // Calls a callback for each subtype and combines the results
// into a final type. // into a final type.
export function doForSubtypes(type: Type, callback: (type: Type) => (Type | undefined)): Type { export function doForSubtypes(type: Type, callback: (type: Type) => Type | undefined): Type {
if (type.category === TypeCategory.Union) { if (type.category === TypeCategory.Union) {
const newTypes: Type[] = []; const newTypes: Type[] = [];
@ -146,17 +161,16 @@ export function stripLiteralTypeArgsValue(type: Type, recursionCount = 0): Type
if (type.category === TypeCategory.Class) { if (type.category === TypeCategory.Class) {
if (type.typeArguments) { if (type.typeArguments) {
const strippedTypeArgs = type.typeArguments.map( const strippedTypeArgs = type.typeArguments.map(t =>
t => stripLiteralTypeArgsValue(stripLiteralValue(t), recursionCount + 1)); stripLiteralTypeArgsValue(stripLiteralValue(t), recursionCount + 1)
return ClassType.cloneForSpecialization(type, strippedTypeArgs, );
type.skipAbstractClassTest); return ClassType.cloneForSpecialization(type, strippedTypeArgs, type.skipAbstractClassTest);
} }
} }
if (type.category === TypeCategory.Object) { if (type.category === TypeCategory.Object) {
if (type.classType.typeArguments) { if (type.classType.typeArguments) {
type = ObjectType.create( type = ObjectType.create(stripLiteralTypeArgsValue(type.classType, recursionCount + 1) as ClassType);
stripLiteralTypeArgsValue(type.classType, recursionCount + 1) as ClassType);
} }
return type; return type;
@ -171,12 +185,12 @@ export function stripLiteralTypeArgsValue(type: Type, recursionCount = 0): Type
if (type.category === TypeCategory.Function) { if (type.category === TypeCategory.Function) {
if (type.specializedTypes) { if (type.specializedTypes) {
const strippedSpecializedTypes: SpecializedFunctionTypes = { const strippedSpecializedTypes: SpecializedFunctionTypes = {
parameterTypes: type.specializedTypes.parameterTypes.map( parameterTypes: type.specializedTypes.parameterTypes.map(t =>
t => stripLiteralTypeArgsValue(stripLiteralValue(t), recursionCount + 1)), stripLiteralTypeArgsValue(stripLiteralValue(t), recursionCount + 1)
returnType: type.specializedTypes.returnType ? ),
stripLiteralTypeArgsValue(stripLiteralValue(type.specializedTypes.returnType), returnType: type.specializedTypes.returnType
recursionCount + 1) : ? stripLiteralTypeArgsValue(stripLiteralValue(type.specializedTypes.returnType), recursionCount + 1)
undefined : undefined
}; };
type = FunctionType.cloneForSpecialization(type, strippedSpecializedTypes); type = FunctionType.cloneForSpecialization(type, strippedSpecializedTypes);
} }
@ -187,7 +201,8 @@ export function stripLiteralTypeArgsValue(type: Type, recursionCount = 0): Type
if (type.category === TypeCategory.OverloadedFunction) { if (type.category === TypeCategory.OverloadedFunction) {
const strippedOverload = OverloadedFunctionType.create(); const strippedOverload = OverloadedFunctionType.create();
strippedOverload.overloads = type.overloads.map( strippedOverload.overloads = type.overloads.map(
t => stripLiteralTypeArgsValue(t, recursionCount + 1) as FunctionType); t => stripLiteralTypeArgsValue(t, recursionCount + 1) as FunctionType
);
return strippedOverload; return strippedOverload;
} }
@ -297,8 +312,7 @@ export function isEllipsisType(type: Type): boolean {
return true; return true;
} }
return (type.category === TypeCategory.Class && return type.category === TypeCategory.Class && ClassType.isBuiltIn(type, 'ellipsis');
ClassType.isBuiltIn(type, 'ellipsis'));
} }
export function isNoReturnType(type: Type): boolean { export function isNoReturnType(type: Type): boolean {
@ -312,8 +326,7 @@ export function isNoReturnType(type: Type): boolean {
} }
export function isProperty(type: Type): boolean { export function isProperty(type: Type): boolean {
return type.category === TypeCategory.Object && return type.category === TypeCategory.Object && ClassType.isPropertyClass(type.classType);
ClassType.isPropertyClass(type.classType);
} }
// Partially specializes a type within the context of a specified // Partially specializes a type within the context of a specified
@ -335,9 +348,12 @@ export function partiallySpecializeType(type: Type, contextClassType: ClassType)
// provided or makeConcrete is true, type variables are replaced // provided or makeConcrete is true, type variables are replaced
// with a concrete type derived from the type variable if there // with a concrete type derived from the type variable if there
// is no corresponding definition in the typeVarMap. // is no corresponding definition in the typeVarMap.
export function specializeType(type: Type, typeVarMap: TypeVarMap | undefined, export function specializeType(
makeConcrete = false, recursionLevel = 0): Type { type: Type,
typeVarMap: TypeVarMap | undefined,
makeConcrete = false,
recursionLevel = 0
): Type {
// Prevent infinite recursion in case a type refers to itself. // Prevent infinite recursion in case a type refers to itself.
if (recursionLevel > 100) { if (recursionLevel > 100) {
return AnyType.create(); return AnyType.create();
@ -379,16 +395,14 @@ export function specializeType(type: Type, typeVarMap: TypeVarMap | undefined,
if (type.category === TypeCategory.Union) { if (type.category === TypeCategory.Union) {
const subtypes: Type[] = []; const subtypes: Type[] = [];
type.subtypes.forEach(typeEntry => { type.subtypes.forEach(typeEntry => {
subtypes.push(specializeType(typeEntry, typeVarMap, subtypes.push(specializeType(typeEntry, typeVarMap, makeConcrete, recursionLevel + 1));
makeConcrete, recursionLevel + 1));
}); });
return combineTypes(subtypes); return combineTypes(subtypes);
} }
if (type.category === TypeCategory.Object) { if (type.category === TypeCategory.Object) {
const classType = _specializeClassType(type.classType, const classType = _specializeClassType(type.classType, typeVarMap, makeConcrete, recursionLevel + 1);
typeVarMap, makeConcrete, recursionLevel + 1);
// Handle the "Type" special class. // Handle the "Type" special class.
if (ClassType.isBuiltIn(classType, 'Type')) { if (ClassType.isBuiltIn(classType, 'Type')) {
@ -396,8 +410,7 @@ export function specializeType(type: Type, typeVarMap: TypeVarMap | undefined,
if (typeArgs && typeArgs.length >= 1) { if (typeArgs && typeArgs.length >= 1) {
const firstTypeArg = typeArgs[0]; const firstTypeArg = typeArgs[0];
if (firstTypeArg.category === TypeCategory.Object) { if (firstTypeArg.category === TypeCategory.Object) {
return specializeType(firstTypeArg.classType, typeVarMap, return specializeType(firstTypeArg.classType, typeVarMap, makeConcrete, recursionLevel + 1);
makeConcrete, recursionLevel + 1);
} else if (firstTypeArg.category === TypeCategory.TypeVar) { } else if (firstTypeArg.category === TypeCategory.TypeVar) {
if (typeVarMap) { if (typeVarMap) {
const replacementType = typeVarMap.get(firstTypeArg.name); const replacementType = typeVarMap.get(firstTypeArg.name);
@ -418,26 +431,26 @@ export function specializeType(type: Type, typeVarMap: TypeVarMap | undefined,
} }
if (type.category === TypeCategory.Class) { if (type.category === TypeCategory.Class) {
return _specializeClassType(type, typeVarMap, return _specializeClassType(type, typeVarMap, makeConcrete, recursionLevel + 1);
makeConcrete, recursionLevel + 1);
} }
if (type.category === TypeCategory.Function) { if (type.category === TypeCategory.Function) {
return _specializeFunctionType(type, typeVarMap, return _specializeFunctionType(type, typeVarMap, makeConcrete, recursionLevel + 1);
makeConcrete, recursionLevel + 1);
} }
if (type.category === TypeCategory.OverloadedFunction) { if (type.category === TypeCategory.OverloadedFunction) {
return _specializeOverloadedFunctionType(type, typeVarMap, return _specializeOverloadedFunctionType(type, typeVarMap, makeConcrete, recursionLevel + 1);
makeConcrete, recursionLevel + 1);
} }
return type; return type;
} }
export function lookUpObjectMember(objectType: Type, memberName: string, importLookup: ImportLookup, export function lookUpObjectMember(
flags = ClassMemberLookupFlags.Default): ClassMember | undefined { objectType: Type,
memberName: string,
importLookup: ImportLookup,
flags = ClassMemberLookupFlags.Default
): ClassMember | undefined {
if (objectType.category === TypeCategory.Object) { if (objectType.category === TypeCategory.Object) {
return lookUpClassMember(objectType.classType, memberName, importLookup, flags); return lookUpClassMember(objectType.classType, memberName, importLookup, flags);
} }
@ -453,9 +466,12 @@ export function lookUpObjectMember(objectType: Type, memberName: string, importL
// ClassB[str] which inherits from Dict[_T1, int], a search for '__iter__' // ClassB[str] which inherits from Dict[_T1, int], a search for '__iter__'
// would return a class type of Dict[str, int] and a symbolType of // would return a class type of Dict[str, int] and a symbolType of
// (self) -> Iterator[str]. // (self) -> Iterator[str].
export function lookUpClassMember(classType: Type, memberName: string, importLookup: ImportLookup, export function lookUpClassMember(
flags = ClassMemberLookupFlags.Default): ClassMember | undefined { classType: Type,
memberName: string,
importLookup: ImportLookup,
flags = ClassMemberLookupFlags.Default
): ClassMember | undefined {
const declaredTypesOnly = (flags & ClassMemberLookupFlags.DeclaredTypesOnly) !== 0; const declaredTypesOnly = (flags & ClassMemberLookupFlags.DeclaredTypesOnly) !== 0;
if (classType.category === TypeCategory.Class) { if (classType.category === TypeCategory.Class) {
@ -515,7 +531,10 @@ export function lookUpClassMember(classType: Type, memberName: string, importLoo
// Recursively perform search. // Recursively perform search.
const methodType = lookUpClassMember( const methodType = lookUpClassMember(
partiallySpecializeType(baseClass, classType), partiallySpecializeType(baseClass, classType),
memberName, importLookup, flags & ~ClassMemberLookupFlags.SkipOriginalClass); memberName,
importLookup,
flags & ~ClassMemberLookupFlags.SkipOriginalClass
);
if (methodType) { if (methodType) {
return methodType; return methodType;
} }
@ -592,8 +611,7 @@ export function getTypeVarArgumentsRecursive(type: Type): TypeVarType[] {
const combinedList: TypeVarType[] = []; const combinedList: TypeVarType[] = [];
if (classType.typeArguments) { if (classType.typeArguments) {
classType.typeArguments.forEach(typeArg => { classType.typeArguments.forEach(typeArg => {
addTypeVarsToListIfUnique(combinedList, addTypeVarsToListIfUnique(combinedList, getTypeVarArgumentsRecursive(typeArg));
getTypeVarArgumentsRecursive(typeArg));
}); });
} }
@ -609,21 +627,18 @@ export function getTypeVarArgumentsRecursive(type: Type): TypeVarType[] {
} else if (type.category === TypeCategory.Union) { } else if (type.category === TypeCategory.Union) {
const combinedList: TypeVarType[] = []; const combinedList: TypeVarType[] = [];
for (const subtype of type.subtypes) { for (const subtype of type.subtypes) {
addTypeVarsToListIfUnique(combinedList, addTypeVarsToListIfUnique(combinedList, getTypeVarArgumentsRecursive(subtype));
getTypeVarArgumentsRecursive(subtype));
} }
return combinedList; return combinedList;
} else if (type.category === TypeCategory.Function) { } else if (type.category === TypeCategory.Function) {
const combinedList: TypeVarType[] = []; const combinedList: TypeVarType[] = [];
type.details.parameters.forEach(param => { type.details.parameters.forEach(param => {
addTypeVarsToListIfUnique(combinedList, addTypeVarsToListIfUnique(combinedList, getTypeVarArgumentsRecursive(param.type));
getTypeVarArgumentsRecursive(param.type));
}); });
if (type.details.declaredReturnType) { if (type.details.declaredReturnType) {
addTypeVarsToListIfUnique(combinedList, addTypeVarsToListIfUnique(combinedList, getTypeVarArgumentsRecursive(type.details.declaredReturnType));
getTypeVarArgumentsRecursive(type.details.declaredReturnType));
} }
return combinedList; return combinedList;
@ -654,9 +669,7 @@ export function stripFirstParameter(type: FunctionType): FunctionType {
// Recursively finds all of the type arguments and sets them // Recursively finds all of the type arguments and sets them
// to the specified srcType. // to the specified srcType.
export function setTypeArgumentsRecursive(destType: Type, srcType: Type, export function setTypeArgumentsRecursive(destType: Type, srcType: Type, typeVarMap: TypeVarMap, recursionCount = 0) {
typeVarMap: TypeVarMap, recursionCount = 0) {
if (typeVarMap.isLocked()) { if (typeVarMap.isLocked()) {
return; return;
} }
@ -686,16 +699,24 @@ export function setTypeArgumentsRecursive(destType: Type, srcType: Type,
setTypeArgumentsRecursive(paramType, srcType, typeVarMap, recursionCount + 1); setTypeArgumentsRecursive(paramType, srcType, typeVarMap, recursionCount + 1);
}); });
if (destType.specializedTypes.returnType) { if (destType.specializedTypes.returnType) {
setTypeArgumentsRecursive(destType.specializedTypes.returnType, srcType, setTypeArgumentsRecursive(
typeVarMap, recursionCount + 1); destType.specializedTypes.returnType,
srcType,
typeVarMap,
recursionCount + 1
);
} }
} else { } else {
destType.details.parameters.forEach(param => { destType.details.parameters.forEach(param => {
setTypeArgumentsRecursive(param.type, srcType, typeVarMap, recursionCount + 1); setTypeArgumentsRecursive(param.type, srcType, typeVarMap, recursionCount + 1);
}); });
if (destType.details.declaredReturnType) { if (destType.details.declaredReturnType) {
setTypeArgumentsRecursive(destType.details.declaredReturnType, srcType, setTypeArgumentsRecursive(
typeVarMap, recursionCount + 1); destType.details.declaredReturnType,
srcType,
typeVarMap,
recursionCount + 1
);
} }
} }
break; break;
@ -836,9 +857,11 @@ export function removeTruthinessFromType(type: Type, importLookup: ImportLookup)
// Looks up the specified symbol name within the base classes // Looks up the specified symbol name within the base classes
// of a specified class. // of a specified class.
export function getSymbolFromBaseClasses(classType: ClassType, name: string, export function getSymbolFromBaseClasses(
recursionCount = 0): SymbolWithClass | undefined { classType: ClassType,
name: string,
recursionCount = 0
): SymbolWithClass | undefined {
if (recursionCount > maxTypeRecursionCount) { if (recursionCount > maxTypeRecursionCount) {
return undefined; return undefined;
} }
@ -867,19 +890,14 @@ export function getSymbolFromBaseClasses(classType: ClassType, name: string,
} }
// Returns the declared yield type if provided, or undefined otherwise. // Returns the declared yield type if provided, or undefined otherwise.
export function getDeclaredGeneratorYieldType(functionType: FunctionType, export function getDeclaredGeneratorYieldType(functionType: FunctionType, iteratorType: Type): Type | undefined {
iteratorType: Type): Type | undefined {
const returnType = FunctionType.getSpecializedReturnType(functionType); const returnType = FunctionType.getSpecializedReturnType(functionType);
if (returnType) { if (returnType) {
const generatorTypeArgs = _getGeneratorReturnTypeArgs(returnType); const generatorTypeArgs = _getGeneratorReturnTypeArgs(returnType);
if (generatorTypeArgs && generatorTypeArgs.length >= 1 && if (generatorTypeArgs && generatorTypeArgs.length >= 1 && iteratorType.category === TypeCategory.Class) {
iteratorType.category === TypeCategory.Class) {
// The yield type is the first type arg. Wrap it in an iterator. // The yield type is the first type arg. Wrap it in an iterator.
return ObjectType.create(ClassType.cloneForSpecialization( return ObjectType.create(ClassType.cloneForSpecialization(iteratorType, [generatorTypeArgs[0]]));
iteratorType, [generatorTypeArgs[0]]));
} }
// If the return type isn't a Generator, assume that it's the // If the return type isn't a Generator, assume that it's the
@ -936,9 +954,7 @@ export function convertClassToObject(type: Type): Type {
}); });
} }
export function getMembersForClass(classType: ClassType, symbolTable: SymbolTable, export function getMembersForClass(classType: ClassType, symbolTable: SymbolTable, includeInstanceVars: boolean) {
includeInstanceVars: boolean) {
_getMembersForClassRecursive(classType, symbolTable, includeInstanceVars); _getMembersForClassRecursive(classType, symbolTable, includeInstanceVars);
} }
@ -957,9 +973,7 @@ export function getMembersForModule(moduleType: ModuleType, symbolTable: SymbolT
}); });
} }
export function containsUnknown(type: Type, allowUnknownTypeArgsForClasses = false, export function containsUnknown(type: Type, allowUnknownTypeArgsForClasses = false, recursionCount = 0): boolean {
recursionCount = 0): boolean {
if (recursionCount > maxTypeRecursionCount) { if (recursionCount > maxTypeRecursionCount) {
return false; return false;
} }
@ -971,9 +985,7 @@ export function containsUnknown(type: Type, allowUnknownTypeArgsForClasses = fal
// See if a union contains an unknown type. // See if a union contains an unknown type.
if (type.category === TypeCategory.Union) { if (type.category === TypeCategory.Union) {
for (const subtype of type.subtypes) { for (const subtype of type.subtypes) {
if (containsUnknown(subtype, allowUnknownTypeArgsForClasses, if (containsUnknown(subtype, allowUnknownTypeArgsForClasses, recursionCount + 1)) {
recursionCount + 1)) {
return true; return true;
} }
} }
@ -989,9 +1001,7 @@ export function containsUnknown(type: Type, allowUnknownTypeArgsForClasses = fal
if (type.category === TypeCategory.Class) { if (type.category === TypeCategory.Class) {
if (type.typeArguments && !allowUnknownTypeArgsForClasses) { if (type.typeArguments && !allowUnknownTypeArgsForClasses) {
for (const argType of type.typeArguments) { for (const argType of type.typeArguments) {
if (containsUnknown(argType, allowUnknownTypeArgsForClasses, if (containsUnknown(argType, allowUnknownTypeArgsForClasses, recursionCount + 1)) {
recursionCount + 1)) {
return true; return true;
} }
} }
@ -1024,18 +1034,19 @@ export function containsUnknown(type: Type, allowUnknownTypeArgsForClasses = fal
return false; return false;
} }
function _getMembersForClassRecursive(classType: ClassType, function _getMembersForClassRecursive(
symbolTable: SymbolTable, includeInstanceVars: boolean, classType: ClassType,
recursionCount = 0) { symbolTable: SymbolTable,
includeInstanceVars: boolean,
recursionCount = 0
) {
if (recursionCount > maxTypeRecursionCount) { if (recursionCount > maxTypeRecursionCount) {
return; return;
} }
classType.details.baseClasses.forEach(baseClassType => { classType.details.baseClasses.forEach(baseClassType => {
if (baseClassType.category === TypeCategory.Class) { if (baseClassType.category === TypeCategory.Class) {
_getMembersForClassRecursive(baseClassType, _getMembersForClassRecursive(baseClassType, symbolTable, includeInstanceVars, recursionCount + 1);
symbolTable, includeInstanceVars, recursionCount + 1);
} }
}); });
@ -1052,9 +1063,12 @@ function _getMembersForClassRecursive(classType: ClassType,
}); });
} }
function _specializeClassType(classType: ClassType, typeVarMap: TypeVarMap | undefined, function _specializeClassType(
makeConcrete: boolean, recursionLevel: number): ClassType { classType: ClassType,
typeVarMap: TypeVarMap | undefined,
makeConcrete: boolean,
recursionLevel: number
): ClassType {
// Handle the common case where the class has no type parameters. // Handle the common case where the class has no type parameters.
if (ClassType.getTypeParameters(classType).length === 0) { if (ClassType.getTypeParameters(classType).length === 0) {
return classType; return classType;
@ -1066,8 +1080,7 @@ function _specializeClassType(classType: ClassType, typeVarMap: TypeVarMap | und
// If type args were previously provided, specialize them. // If type args were previously provided, specialize them.
if (classType.typeArguments) { if (classType.typeArguments) {
newTypeArgs = classType.typeArguments.map(oldTypeArgType => { newTypeArgs = classType.typeArguments.map(oldTypeArgType => {
const newTypeArgType = specializeType(oldTypeArgType, const newTypeArgType = specializeType(oldTypeArgType, typeVarMap, makeConcrete, recursionLevel + 1);
typeVarMap, makeConcrete, recursionLevel + 1);
if (newTypeArgType !== oldTypeArgType) { if (newTypeArgType !== oldTypeArgType) {
specializationNeeded = true; specializationNeeded = true;
} }
@ -1118,13 +1131,16 @@ export function getConcreteTypeFromTypeVar(type: TypeVarType, recursionLevel = 0
return UnknownType.create(); return UnknownType.create();
} }
function _specializeOverloadedFunctionType(type: OverloadedFunctionType, function _specializeOverloadedFunctionType(
typeVarMap: TypeVarMap | undefined, makeConcrete: boolean, type: OverloadedFunctionType,
recursionLevel: number): OverloadedFunctionType { typeVarMap: TypeVarMap | undefined,
makeConcrete: boolean,
recursionLevel: number
): OverloadedFunctionType {
// Specialize each of the functions in the overload. // Specialize each of the functions in the overload.
const overloads = type.overloads.map( const overloads = type.overloads.map(entry =>
entry => _specializeFunctionType(entry, typeVarMap, makeConcrete, recursionLevel)); _specializeFunctionType(entry, typeVarMap, makeConcrete, recursionLevel)
);
// Construct a new overload with the specialized function types. // Construct a new overload with the specialized function types.
const newOverloadType = OverloadedFunctionType.create(); const newOverloadType = OverloadedFunctionType.create();
@ -1135,15 +1151,19 @@ function _specializeOverloadedFunctionType(type: OverloadedFunctionType,
return newOverloadType; return newOverloadType;
} }
function _specializeFunctionType(functionType: FunctionType, function _specializeFunctionType(
typeVarMap: TypeVarMap | undefined, makeConcrete: boolean, functionType: FunctionType,
recursionLevel: number): FunctionType { typeVarMap: TypeVarMap | undefined,
makeConcrete: boolean,
const declaredReturnType = functionType.specializedTypes && functionType.specializedTypes.returnType ? recursionLevel: number
functionType.specializedTypes.returnType : functionType.details.declaredReturnType; ): FunctionType {
const specializedReturnType = declaredReturnType ? const declaredReturnType =
specializeType(declaredReturnType, typeVarMap, makeConcrete, recursionLevel + 1) : functionType.specializedTypes && functionType.specializedTypes.returnType
undefined; ? functionType.specializedTypes.returnType
: functionType.details.declaredReturnType;
const specializedReturnType = declaredReturnType
? specializeType(declaredReturnType, typeVarMap, makeConcrete, recursionLevel + 1)
: undefined;
let typesRequiredSpecialization = declaredReturnType !== specializedReturnType; let typesRequiredSpecialization = declaredReturnType !== specializedReturnType;
const specializedParameters: SpecializedFunctionTypes = { const specializedParameters: SpecializedFunctionTypes = {
@ -1153,8 +1173,7 @@ function _specializeFunctionType(functionType: FunctionType,
for (let i = 0; i < functionType.details.parameters.length; i++) { for (let i = 0; i < functionType.details.parameters.length; i++) {
const paramType = FunctionType.getEffectiveParameterType(functionType, i); const paramType = FunctionType.getEffectiveParameterType(functionType, i);
const specializedType = specializeType(paramType, const specializedType = specializeType(paramType, typeVarMap, makeConcrete, recursionLevel + 1);
typeVarMap, makeConcrete, recursionLevel + 1);
specializedParameters.parameterTypes.push(specializedType); specializedParameters.parameterTypes.push(specializedType);
if (paramType !== specializedType) { if (paramType !== specializedType) {
@ -1197,9 +1216,10 @@ export function requiresSpecialization(type: Type, recursionCount = 0): boolean
return false; return false;
} }
return type.typeArguments.find( return (
typeArg => requiresSpecialization(typeArg, recursionCount + 1) type.typeArguments.find(typeArg => requiresSpecialization(typeArg, recursionCount + 1)) !==
) !== undefined; undefined
);
} }
// If there are any type parameters, we need to specialize // If there are any type parameters, we need to specialize
@ -1220,14 +1240,16 @@ export function requiresSpecialization(type: Type, recursionCount = 0): boolean
return false; return false;
} }
for (let i = 0; i < type.details.parameters.length; i ++) { for (let i = 0; i < type.details.parameters.length; i++) {
if (requiresSpecialization(FunctionType.getEffectiveParameterType(type, i), recursionCount + 1)) { if (requiresSpecialization(FunctionType.getEffectiveParameterType(type, i), recursionCount + 1)) {
return true; return true;
} }
} }
const declaredReturnType = type.specializedTypes && type.specializedTypes.returnType ? const declaredReturnType =
type.specializedTypes.returnType : type.details.declaredReturnType; type.specializedTypes && type.specializedTypes.returnType
? type.specializedTypes.returnType
: type.details.declaredReturnType;
if (declaredReturnType) { if (declaredReturnType) {
if (requiresSpecialization(declaredReturnType, recursionCount + 1)) { if (requiresSpecialization(declaredReturnType, recursionCount + 1)) {
return true; return true;
@ -1238,13 +1260,11 @@ export function requiresSpecialization(type: Type, recursionCount = 0): boolean
} }
case TypeCategory.OverloadedFunction: { case TypeCategory.OverloadedFunction: {
return type.overloads.find( return type.overloads.find(overload => requiresSpecialization(overload, recursionCount + 1)) !== undefined;
overload => requiresSpecialization(overload, recursionCount + 1)) !== undefined;
} }
case TypeCategory.Union: { case TypeCategory.Union: {
return type.subtypes.find( return type.subtypes.find(type => requiresSpecialization(type, recursionCount + 1)) !== undefined;
type => requiresSpecialization(type, recursionCount + 1)) !== undefined;
} }
case TypeCategory.TypeVar: { case TypeCategory.TypeVar: {
@ -1262,10 +1282,10 @@ export function printLiteralValue(type: ObjectType): string {
} }
let literalStr: string; let literalStr: string;
if (typeof(literalValue) === 'string') { if (typeof literalValue === 'string') {
const prefix = (type.classType.details.name === 'bytes') ? 'b' : ''; const prefix = type.classType.details.name === 'bytes' ? 'b' : '';
literalStr = `${ prefix }'${ literalValue.toString() }'`; literalStr = `${prefix}'${literalValue.toString()}'`;
} else if (typeof(literalValue) === 'boolean') { } else if (typeof literalValue === 'boolean') {
literalStr = literalValue ? 'True' : 'False'; literalStr = literalValue ? 'True' : 'False';
} else { } else {
literalStr = literalValue.toString(); literalStr = literalValue.toString();
@ -1280,5 +1300,5 @@ export function printLiteralType(type: ObjectType): string {
return ''; return '';
} }
return `Literal[${ literalStr }]`; return `Literal[${literalStr}]`;
} }

22
server/src/analyzer/typeVarMap.ts
View File

@ -1,16 +1,16 @@
/* /*
* typeVarMap.ts * typeVarMap.ts
* Copyright (c) Microsoft Corporation. * Copyright (c) Microsoft Corporation.
* Licensed under the MIT license. * Licensed under the MIT license.
* Author: Eric Traut * Author: Eric Traut
* *
* Module that records the relationship between named TypeVars * Module that records the relationship between named TypeVars
* (type variables) and their types. It is used by the type * (type variables) and their types. It is used by the type
* evaluator to "solve" for the type of each type variable. * evaluator to "solve" for the type of each type variable.
*/ */
import { assert } from "../common/debug"; import { assert } from '../common/debug';
import { Type } from "./types"; import { Type } from './types';
export class TypeVarMap { export class TypeVarMap {
private _typeMap: Map<string, Type>; private _typeMap: Map<string, Type>;

204
server/src/analyzer/types.ts
View File

@ -1,11 +1,11 @@
/* /*
* types.ts * types.ts
* Copyright (c) Microsoft Corporation. * Copyright (c) Microsoft Corporation.
* Licensed under the MIT license. * Licensed under the MIT license.
* Author: Eric Traut * Author: Eric Traut
* *
* Representation of types used during type analysis within Python. * Representation of types used during type analysis within Python.
*/ */
import { assert } from '../common/debug'; import { assert } from '../common/debug';
import { ParameterCategory } from '../parser/parseNodes'; import { ParameterCategory } from '../parser/parseNodes';
@ -56,9 +56,19 @@ export const enum TypeCategory {
TypeVar TypeVar
} }
export type Type = UnboundType | UnknownType | AnyType | NoneType | NeverType | export type Type =
FunctionType | OverloadedFunctionType | ClassType | | UnboundType
ObjectType | ModuleType | UnionType | TypeVarType; | UnknownType
| AnyType
| NoneType
| NeverType
| FunctionType
| OverloadedFunctionType
| ClassType
| ObjectType
| ModuleType
| UnionType
| TypeVarType;
export type LiteralValue = number | boolean | string; export type LiteralValue = number | boolean | string;
@ -145,34 +155,34 @@ export const enum ClassTypeFlags {
None = 0, None = 0,
// Class is defined in the "builtins" or "typings" file. // Class is defined in the "builtins" or "typings" file.
BuiltInClass = 1 << 0, BuiltInClass = 1 << 0,
// Class requires special-case handling because it // Class requires special-case handling because it
// exhibits non-standard behavior or is not defined // exhibits non-standard behavior or is not defined
// formally as a class. Examples include 'Optional' // formally as a class. Examples include 'Optional'
// and 'Union'. // and 'Union'.
SpecialBuiltIn = 1 << 1, SpecialBuiltIn = 1 << 1,
// Introduced in Python 3.7 - class either derives directly // Introduced in Python 3.7 - class either derives directly
// from NamedTuple or has a @dataclass class decorator. // from NamedTuple or has a @dataclass class decorator.
DataClass = 1 << 2, DataClass = 1 << 2,
// Flags that control whether methods should be // Flags that control whether methods should be
// synthesized for a dataclass class. // synthesized for a dataclass class.
SkipSynthesizedInit = 1 << 3, SkipSynthesizedInit = 1 << 3,
// Introduced in PEP 589, TypedDict classes provide a way // Introduced in PEP 589, TypedDict classes provide a way
// to specify type hints for dictionaries with different // to specify type hints for dictionaries with different
// value types and a limited set of static keys. // value types and a limited set of static keys.
TypedDictClass = 1 << 4, TypedDictClass = 1 << 4,
// Used in conjunction with TypedDictClass, indicates that // Used in conjunction with TypedDictClass, indicates that
// the dictionary values can be omitted. // the dictionary values can be omitted.
CanOmitDictValues = 1 << 5, CanOmitDictValues = 1 << 5,
// The class has a metaclass of EnumMet or derives from // The class has a metaclass of EnumMet or derives from
// a class that has this metaclass. // a class that has this metaclass.
EnumClass = 1 << 6, EnumClass = 1 << 6,
// The class derives from a class that has the ABCMeta // The class derives from a class that has the ABCMeta
// metaclass. Such classes are allowed to contain // metaclass. Such classes are allowed to contain
@ -182,18 +192,18 @@ export const enum ClassTypeFlags {
// The class has at least one abstract method or derives // The class has at least one abstract method or derives
// from a base class that is abstract without providing // from a base class that is abstract without providing
// non-abstract overrides for all abstract methods. // non-abstract overrides for all abstract methods.
HasAbstractMethods = 1 << 8, HasAbstractMethods = 1 << 8,
// Derives from property class and has the semantics of // Derives from property class and has the semantics of
// a property (with optional setter, deleter). // a property (with optional setter, deleter).
PropertyClass = 1 << 9, PropertyClass = 1 << 9,
// The class is decorated with a "@final" decorator // The class is decorated with a "@final" decorator
// indicating that it cannot be subclassed. // indicating that it cannot be subclassed.
Final = 1 << 10, Final = 1 << 10,
// The class derives directly from "Protocol". // The class derives directly from "Protocol".
ProtocolClass = 1 << 11 ProtocolClass = 1 << 11
} }
interface ClassDetails { interface ClassDetails {
@ -224,9 +234,7 @@ export interface ClassType extends TypeBase {
} }
export namespace ClassType { export namespace ClassType {
export function create(name: string, flags: ClassTypeFlags, export function create(name: string, flags: ClassTypeFlags, typeSourceId: TypeSourceId, docString?: string) {
typeSourceId: TypeSourceId, docString?: string) {
const newClass: ClassType = { const newClass: ClassType = {
category: TypeCategory.Class, category: TypeCategory.Class,
details: { details: {
@ -244,11 +252,12 @@ export namespace ClassType {
return newClass; return newClass;
} }
export function cloneForSpecialization(classType: ClassType, export function cloneForSpecialization(
typeArguments: Type[] | undefined, skipAbstractClassTest = false): ClassType { classType: ClassType,
typeArguments: Type[] | undefined,
const newClassType = create(classType.details.name, skipAbstractClassTest = false
classType.details.flags, classType.details.typeSourceId); ): ClassType {
const newClassType = create(classType.details.name, classType.details.flags, classType.details.typeSourceId);
newClassType.details = classType.details; newClassType.details = classType.details;
newClassType.typeArguments = typeArguments; newClassType.typeArguments = typeArguments;
if (skipAbstractClassTest) { if (skipAbstractClassTest) {
@ -260,8 +269,7 @@ export namespace ClassType {
// Specifies whether the class type is generic (unspecialized) // Specifies whether the class type is generic (unspecialized)
// or specialized. // or specialized.
export function isGeneric(classType: ClassType) { export function isGeneric(classType: ClassType) {
return classType.details.typeParameters.length > 0 && return classType.details.typeParameters.length > 0 && classType.typeArguments === undefined;
classType.typeArguments === undefined;
} }
export function isSpecialBuiltIn(classType: ClassType, className?: string) { export function isSpecialBuiltIn(classType: ClassType, className?: string) {
@ -290,19 +298,20 @@ export namespace ClassType {
export function isProtocol(classType: ClassType) { export function isProtocol(classType: ClassType) {
// Does the class directly 'derive' from "Protocol"? // Does the class directly 'derive' from "Protocol"?
return classType.details.baseClasses.find(baseClass => { return (
if (baseClass.category === TypeCategory.Class) { classType.details.baseClasses.find(baseClass => {
if (isBuiltIn(baseClass, 'Protocol')) { if (baseClass.category === TypeCategory.Class) {
return true; if (isBuiltIn(baseClass, 'Protocol')) {
return true;
}
} }
} return false;
return false; }) !== undefined
}) !== undefined; );
} }
export function hasAbstractMethods(classType: ClassType) { export function hasAbstractMethods(classType: ClassType) {
return !!(classType.details.flags & ClassTypeFlags.HasAbstractMethods) && return !!(classType.details.flags & ClassTypeFlags.HasAbstractMethods) && !classType.skipAbstractClassTest;
!classType.skipAbstractClassTest;
} }
export function supportsAbstractMethods(classType: ClassType) { export function supportsAbstractMethods(classType: ClassType) {
@ -367,10 +376,8 @@ export namespace ClassType {
// If either or both have aliases (e.g. List -> list), use the // If either or both have aliases (e.g. List -> list), use the
// aliases for comparison purposes. // aliases for comparison purposes.
const class1Details = classType.details.aliasClass ? const class1Details = classType.details.aliasClass ? classType.details.aliasClass.details : classType.details;
classType.details.aliasClass.details : classType.details; const class2Details = type2.details.aliasClass ? type2.details.aliasClass.details : type2.details;
const class2Details = type2.details.aliasClass ?
type2.details.aliasClass.details : type2.details;
if (class1Details === class2Details) { if (class1Details === class2Details) {
return true; return true;
@ -378,34 +385,34 @@ export namespace ClassType {
// Compare most of the details fields. We intentionally skip the isAbstractClass // Compare most of the details fields. We intentionally skip the isAbstractClass
// flag because it gets set dynamically. // flag because it gets set dynamically.
if (class1Details.name !== class2Details.name || if (
class1Details.flags !== class2Details.flags || class1Details.name !== class2Details.name ||
class1Details.typeSourceId !== class2Details.typeSourceId || class1Details.flags !== class2Details.flags ||
class1Details.baseClasses.length !== class2Details.baseClasses.length || class1Details.typeSourceId !== class2Details.typeSourceId ||
class1Details.typeParameters.length !== class2Details.typeParameters.length) { class1Details.baseClasses.length !== class2Details.baseClasses.length ||
class1Details.typeParameters.length !== class2Details.typeParameters.length
) {
return false; return false;
} }
for (let i = 0; i < class1Details.baseClasses.length; i++) { for (let i = 0; i < class1Details.baseClasses.length; i++) {
if (!isTypeSame(class1Details.baseClasses[i], class2Details.baseClasses[i], if (!isTypeSame(class1Details.baseClasses[i], class2Details.baseClasses[i], recursionCount + 1)) {
recursionCount + 1)) {
return false; return false;
} }
} }
if (class1Details.metaClass || class2Details.metaClass) { if (class1Details.metaClass || class2Details.metaClass) {
if (!class1Details.metaClass || !class2Details.metaClass || if (
!isTypeSame(class1Details.metaClass, class2Details.metaClass)) { !class1Details.metaClass ||
!class2Details.metaClass ||
!isTypeSame(class1Details.metaClass, class2Details.metaClass)
) {
return false; return false;
} }
} }
for (let i = 0; i < class1Details.typeParameters.length; i++) { for (let i = 0; i < class1Details.typeParameters.length; i++) {
if (!isTypeSame(class1Details.typeParameters[i], class2Details.typeParameters[i], if (!isTypeSame(class1Details.typeParameters[i], class2Details.typeParameters[i], recursionCount + 1)) {
recursionCount + 1)) {
return false; return false;
} }
} }
@ -417,10 +424,11 @@ export namespace ClassType {
} }
for (let i = 0; i < dataClassEntries1.length; i++) { for (let i = 0; i < dataClassEntries1.length; i++) {
if (dataClassEntries1[i].name !== dataClassEntries2[i].name || if (
dataClassEntries1[i].hasDefault !== dataClassEntries2[i].hasDefault || dataClassEntries1[i].name !== dataClassEntries2[i].name ||
!isTypeSame(dataClassEntries1[i].type, dataClassEntries2[i].type, recursionCount + 1)) { dataClassEntries1[i].hasDefault !== dataClassEntries2[i].hasDefault ||
!isTypeSame(dataClassEntries1[i].type, dataClassEntries2[i].type, recursionCount + 1)
) {
return false; return false;
} }
} }
@ -460,9 +468,11 @@ export namespace ClassType {
// array to inheritanceChain, it will be filled in by // array to inheritanceChain, it will be filled in by
// the call to include the chain of inherited classes starting // the call to include the chain of inherited classes starting
// with type2 and ending with this type. // with type2 and ending with this type.
export function isDerivedFrom(subclassType: ClassType, export function isDerivedFrom(
parentClassType: ClassType, inheritanceChain?: InheritanceChain): boolean { subclassType: ClassType,
parentClassType: ClassType,
inheritanceChain?: InheritanceChain
): boolean {
// Is it the exact same class? // Is it the exact same class?
if (isSameGenericClass(subclassType, parentClassType)) { if (isSameGenericClass(subclassType, parentClassType)) {
if (inheritanceChain) { if (inheritanceChain) {
@ -537,55 +547,55 @@ export interface FunctionParameter {
} }
export const enum FunctionTypeFlags { export const enum FunctionTypeFlags {
None = 0, None = 0,
// Function is a __new__ method; first parameter is "cls" // Function is a __new__ method; first parameter is "cls"
ConstructorMethod = 1 << 0, ConstructorMethod = 1 << 0,
// Function is decorated with @classmethod; first parameter is "cls"; // Function is decorated with @classmethod; first parameter is "cls";
// can be bound to associated class // can be bound to associated class
ClassMethod = 1 << 1, ClassMethod = 1 << 1,
// Function is decorated with @staticmethod; cannot be bound to class // Function is decorated with @staticmethod; cannot be bound to class
StaticMethod = 1 << 2, StaticMethod = 1 << 2,
// Function is decorated with @abstractmethod // Function is decorated with @abstractmethod
AbstractMethod = 1 << 3, AbstractMethod = 1 << 3,
// Function contains "yield" or "yield from" statements // Function contains "yield" or "yield from" statements
Generator = 1 << 4, Generator = 1 << 4,
// Skip check that validates that all parameters without default // Skip check that validates that all parameters without default
// value expressions have corresponding arguments; used for // value expressions have corresponding arguments; used for
// named tuples in some cases // named tuples in some cases
DisableDefaultChecks = 1 << 5, DisableDefaultChecks = 1 << 5,
// Method has no declaration in user code, it's synthesized; used // Method has no declaration in user code, it's synthesized; used
// for implied methods such as those used in namedtuple, dataclass, etc. // for implied methods such as those used in namedtuple, dataclass, etc.
SynthesizedMethod = 1 << 6, SynthesizedMethod = 1 << 6,
// For some synthesized classes (in particular, NamedTuple), the // For some synthesized classes (in particular, NamedTuple), the
// __init__ method is created with default parameters, so we will // __init__ method is created with default parameters, so we will
// skip the constructor check for these methods. // skip the constructor check for these methods.
SkipConstructorCheck = 1 << 7, SkipConstructorCheck = 1 << 7,
// Function is decorated with @overload // Function is decorated with @overload
Overloaded = 1 << 8, Overloaded = 1 << 8,
// Function is declared with async keyword // Function is declared with async keyword
Async = 1 << 9, Async = 1 << 9,
// Indicates that return type should be wrapped in an awaitable type // Indicates that return type should be wrapped in an awaitable type
WrapReturnTypeInAwait = 1 << 10, WrapReturnTypeInAwait = 1 << 10,
// Function is declared within a type stub fille // Function is declared within a type stub fille
StubDefinition = 1 << 11, StubDefinition = 1 << 11,
// Function is decorated with @final // Function is decorated with @final
Final = 1 << 12, Final = 1 << 12,
// Function has one or more parameters that are missing type annotations // Function has one or more parameters that are missing type annotations
UnannotatedParams = 1 << 13 UnannotatedParams = 1 << 13
} }
interface FunctionDetails { interface FunctionDetails {
@ -650,8 +660,7 @@ export namespace FunctionType {
// If we strip off the first parameter, this is no longer an // If we strip off the first parameter, this is no longer an
// instance method or class method. // instance method or class method.
if (deleteFirstParam) { if (deleteFirstParam) {
newFunction.details.flags &= ~(FunctionTypeFlags.ConstructorMethod | newFunction.details.flags &= ~(FunctionTypeFlags.ConstructorMethod | FunctionTypeFlags.ClassMethod);
FunctionTypeFlags.ClassMethod);
newFunction.details.flags |= FunctionTypeFlags.StaticMethod; newFunction.details.flags |= FunctionTypeFlags.StaticMethod;
newFunction.ignoreFirstParamOfDeclaration = true; newFunction.ignoreFirstParamOfDeclaration = true;
} }
@ -671,9 +680,10 @@ export namespace FunctionType {
// Creates a shallow copy of the function type with new // Creates a shallow copy of the function type with new
// specialized types. The clone shares the _functionDetails // specialized types. The clone shares the _functionDetails
// with the object being cloned. // with the object being cloned.
export function cloneForSpecialization(type: FunctionType, export function cloneForSpecialization(
specializedTypes: SpecializedFunctionTypes): FunctionType { type: FunctionType,
specializedTypes: SpecializedFunctionTypes
): FunctionType {
const newFunction = create(type.details.flags, type.details.docString); const newFunction = create(type.details.flags, type.details.docString);
newFunction.details = type.details; newFunction.details = type.details;
@ -684,8 +694,13 @@ export namespace FunctionType {
} }
export function isInstanceMethod(type: FunctionType): boolean { export function isInstanceMethod(type: FunctionType): boolean {
return (type.details.flags & (FunctionTypeFlags.ConstructorMethod | return (
FunctionTypeFlags.StaticMethod | FunctionTypeFlags.ClassMethod)) === 0; (type.details.flags &
(FunctionTypeFlags.ConstructorMethod |
FunctionTypeFlags.StaticMethod |
FunctionTypeFlags.ClassMethod)) ===
0
);
} }
export function isConstructorMethod(type: FunctionType): boolean { export function isConstructorMethod(type: FunctionType): boolean {
@ -758,8 +773,9 @@ export namespace FunctionType {
} }
export function getSpecializedReturnType(type: FunctionType) { export function getSpecializedReturnType(type: FunctionType) {
return type.specializedTypes && type.specializedTypes.returnType ? return type.specializedTypes && type.specializedTypes.returnType
type.specializedTypes.returnType : type.details.declaredReturnType; ? type.specializedTypes.returnType
: type.details.declaredReturnType;
} }
} }
@ -886,13 +902,11 @@ export namespace TypeVarType {
} }
export function isNoneOrNever(type: Type): boolean { export function isNoneOrNever(type: Type): boolean {
return type.category === TypeCategory.None || return type.category === TypeCategory.None || type.category === TypeCategory.Never;
type.category === TypeCategory.Never;
} }
export function isAnyOrUnknown(type: Type): boolean { export function isAnyOrUnknown(type: Type): boolean {
if (type.category === TypeCategory.Any || if (type.category === TypeCategory.Any || type.category === TypeCategory.Unknown) {
type.category === TypeCategory.Unknown) {
return true; return true;
} }
@ -1005,9 +1019,7 @@ export function isTypeSame(type1: Type, type2: Type, recursionCount = 0): boolea
return2Type = functionType2.specializedTypes.returnType; return2Type = functionType2.specializedTypes.returnType;
} }
if (return1Type || return2Type) { if (return1Type || return2Type) {
if (!return1Type || !return2Type || if (!return1Type || !return2Type || !isTypeSame(return1Type, return2Type, recursionCount + 1)) {
!isTypeSame(return1Type, return2Type, recursionCount + 1)) {
return false; return false;
} }
} }

14
server/src/commands/commands.ts
View File

@ -1,11 +1,11 @@
/* /*
* commands.ts * commands.ts
* Copyright (c) Microsoft Corporation. * Copyright (c) Microsoft Corporation.
* Licensed under the MIT license. * Licensed under the MIT license.
* Author: Eric Traut * Author: Eric Traut
* *
* Command identifier strings. * Command identifier strings.
*/ */
export const enum Commands { export const enum Commands {
createTypeStub = 'pyright.createtypestub', createTypeStub = 'pyright.createtypestub',

8
server/src/commands/createTypeStub.ts
View File

@ -13,7 +13,7 @@ import { AnalyzerServiceExecutor } from '../languageService/analyzerServiceExecu
import { ServerCommand } from './commandController'; import { ServerCommand } from './commandController';
export class CreateTypeStubCommand implements ServerCommand { export class CreateTypeStubCommand implements ServerCommand {
constructor(private _ls: LanguageServerInterface) { } constructor(private _ls: LanguageServerInterface) {}
async execute(cmdParams: ExecuteCommandParams): Promise<any> { async execute(cmdParams: ExecuteCommandParams): Promise<any> {
if (cmdParams.arguments && cmdParams.arguments.length >= 2) { if (cmdParams.arguments && cmdParams.arguments.length >= 2) {
@ -25,7 +25,7 @@ export class CreateTypeStubCommand implements ServerCommand {
// Allocate a temporary pseudo-workspace to perform this job. // Allocate a temporary pseudo-workspace to perform this job.
const workspace: WorkspaceServiceInstance = { const workspace: WorkspaceServiceInstance = {
workspaceName: `Create Type Stub ${ importName }`, workspaceName: `Create Type Stub ${importName}`,
rootPath: workspaceRoot, rootPath: workspaceRoot,
rootUri: convertPathToUri(workspaceRoot), rootUri: convertPathToUri(workspaceRoot),
serviceInstance: service, serviceInstance: service,
@ -37,7 +37,7 @@ export class CreateTypeStubCommand implements ServerCommand {
try { try {
service.writeTypeStub(); service.writeTypeStub();
service.dispose(); service.dispose();
const infoMessage = `Type stub was successfully created for '${ importName }'.`; const infoMessage = `Type stub was successfully created for '${importName}'.`;
this._ls.window.showInformationMessage(infoMessage); this._ls.window.showInformationMessage(infoMessage);
this._handlePostCreateTypeStub(); this._handlePostCreateTypeStub();
} catch (err) { } catch (err) {
@ -45,7 +45,7 @@ export class CreateTypeStubCommand implements ServerCommand {
if (err instanceof Error) { if (err instanceof Error) {
errMessage = ': ' + err.message; errMessage = ': ' + err.message;
} }
errMessage = `An error occurred when creating type stub for '${ importName }'` + errMessage; errMessage = `An error occurred when creating type stub for '${importName}'` + errMessage;
this._ls.console.error(errMessage); this._ls.console.error(errMessage);
this._ls.window.showErrorMessage(errMessage); this._ls.window.showErrorMessage(errMessage);
} }

2
server/src/commands/quickActionCommand.ts
View File

@ -11,7 +11,7 @@ import { LanguageServerInterface } from '../languageServerBase';
import { ServerCommand } from './commandController'; import { ServerCommand } from './commandController';
export class QuickActionCommand implements ServerCommand { export class QuickActionCommand implements ServerCommand {
constructor(private _ls: LanguageServerInterface) { } constructor(private _ls: LanguageServerInterface) {}
async execute(cmdParams: ExecuteCommandParams): Promise<any> { async execute(cmdParams: ExecuteCommandParams): Promise<any> {
if (cmdParams.arguments && cmdParams.arguments.length >= 1) { if (cmdParams.arguments && cmdParams.arguments.length >= 1) {

75
server/src/common/collectionUtils.ts
View File

@ -11,7 +11,11 @@ import { compareValues, Comparison, equateValues, isArray } from './core';
export const emptyArray: never[] = [] as never[]; export const emptyArray: never[] = [] as never[];
export type EqualityComparer<T> = (a: T, b: T) => boolean; export type EqualityComparer<T> = (a: T, b: T) => boolean;
export function contains<T>(array: readonly T[] | undefined, value: T, equalityComparer: EqualityComparer<T> = equateValues): boolean { export function contains<T>(
array: readonly T[] | undefined,
value: T,
equalityComparer: EqualityComparer<T> = equateValues
): boolean {
if (array) { if (array) {
for (const v of array) { for (const v of array) {
if (equalityComparer(v, value)) { if (equalityComparer(v, value)) {
@ -36,20 +40,29 @@ export interface Push<T> {
* appended. * appended.
*/ */
export function append<TArray extends any[] | undefined, TValue extends NonNullable<TArray>[number] | undefined>( export function append<TArray extends any[] | undefined, TValue extends NonNullable<TArray>[number] | undefined>(
to: TArray, value: TValue): [undefined, undefined] extends [TArray, TValue] ? TArray : NonNullable<TArray>[number][]; to: TArray,
value: TValue
): [undefined, undefined] extends [TArray, TValue] ? TArray : NonNullable<TArray>[number][];
export function append<T>(to: T[], value: T | undefined): T[]; export function append<T>(to: T[], value: T | undefined): T[];
export function append<T>(to: T[] | undefined, value: T): T[]; export function append<T>(to: T[] | undefined, value: T): T[];
export function append<T>(to: T[] | undefined, value: T | undefined): T[] | undefined; export function append<T>(to: T[] | undefined, value: T | undefined): T[] | undefined;
export function append<T>(to: Push<T>, value: T | undefined): void; export function append<T>(to: Push<T>, value: T | undefined): void;
export function append<T>(to: T[], value: T | undefined): T[] | undefined { export function append<T>(to: T[], value: T | undefined): T[] | undefined {
if (value === undefined) { return to; } if (value === undefined) {
if (to === undefined) { return [value]; } return to;
}
if (to === undefined) {
return [value];
}
to.push(value); to.push(value);
return to; return to;
} }
/** Works like Array.prototype.find, returning `undefined` if no element satisfying the predicate is found. */ /** Works like Array.prototype.find, returning `undefined` if no element satisfying the predicate is found. */
export function find<T, U extends T>(array: readonly T[], predicate: (element: T, index: number) => element is U): U | undefined; export function find<T, U extends T>(
array: readonly T[],
predicate: (element: T, index: number) => element is U
): U | undefined;
export function find<T>(array: readonly T[], predicate: (element: T, index: number) => boolean): T | undefined; export function find<T>(array: readonly T[], predicate: (element: T, index: number) => boolean): T | undefined;
export function find<T>(array: readonly T[], predicate: (element: T, index: number) => boolean): T | undefined { export function find<T>(array: readonly T[], predicate: (element: T, index: number) => boolean): T | undefined {
for (let i = 0; i < array.length; i++) { for (let i = 0; i < array.length; i++) {
@ -80,10 +93,24 @@ function toOffset(array: readonly any[], offset: number) {
* @param end The offset in `from` at which to stop copying values (non-inclusive). * @param end The offset in `from` at which to stop copying values (non-inclusive).
*/ */
export function addRange<T>(to: T[], from: readonly T[] | undefined, start?: number, end?: number): T[]; export function addRange<T>(to: T[], from: readonly T[] | undefined, start?: number, end?: number): T[];
export function addRange<T>(to: T[] | undefined, from: readonly T[] | undefined, start?: number, end?: number): T[] | undefined; export function addRange<T>(
export function addRange<T>(to: T[] | undefined, from: readonly T[] | undefined, start?: number, end?: number): T[] | undefined { to: T[] | undefined,
if (from === undefined || from.length === 0) { return to; } from: readonly T[] | undefined,
if (to === undefined) { return from.slice(start, end); } start?: number,
end?: number
): T[] | undefined;
export function addRange<T>(
to: T[] | undefined,
from: readonly T[] | undefined,
start?: number,
end?: number
): T[] | undefined {
if (from === undefined || from.length === 0) {
return to;
}
if (to === undefined) {
return from.slice(start, end);
}
start = start === undefined ? 0 : toOffset(from, start); start = start === undefined ? 0 : toOffset(from, start);
end = end === undefined ? from.length : toOffset(from, end); end = end === undefined ? from.length : toOffset(from, end);
for (let i = start; i < end && i < from.length; i++) { for (let i = start; i < end && i < from.length; i++) {
@ -139,7 +166,7 @@ function indicesOf(array: readonly unknown[]): number[] {
export function stableSort<T>(array: readonly T[], comparer: Comparer<T>): SortedReadonlyArray<T> { export function stableSort<T>(array: readonly T[], comparer: Comparer<T>): SortedReadonlyArray<T> {
const indices = indicesOf(array); const indices = indicesOf(array);
stableSortIndices(array, indices, comparer); stableSortIndices(array, indices, comparer);
return indices.map(i => array[i]) as SortedArray<T> as SortedReadonlyArray<T>; return (indices.map(i => array[i]) as SortedArray<T>) as SortedReadonlyArray<T>;
} }
function stableSortIndices<T>(array: readonly T[], indices: number[], comparer: Comparer<T>) { function stableSortIndices<T>(array: readonly T[], indices: number[], comparer: Comparer<T>) {
@ -170,10 +197,10 @@ export function some<T>(array: readonly T[] | undefined, predicate?: (value: T)
} }
/** /**
* Iterates through `array` by index and performs the callback on each element of array until the callback * Iterates through `array` by index and performs the callback on each element of array until the callback
* returns a falsey value, then returns false. * returns a falsey value, then returns false.
* If no such value is found, the callback is applied to each element of array and `true` is returned. * If no such value is found, the callback is applied to each element of array and `true` is returned.
*/ */
export function every<T>(array: readonly T[], callback: (element: T, index: number) => boolean): boolean { export function every<T>(array: readonly T[], callback: (element: T, index: number) => boolean): boolean {
if (array) { if (array) {
return array.every(callback); return array.every(callback);
@ -193,9 +220,13 @@ export function every<T>(array: readonly T[], callback: (element: T, index: numb
* @param keyComparer A callback used to compare two keys in a sorted array. * @param keyComparer A callback used to compare two keys in a sorted array.
* @param offset An offset into `array` at which to start the search. * @param offset An offset into `array` at which to start the search.
*/ */
export function binarySearch<T, U>(array: readonly T[], value: T, export function binarySearch<T, U>(
keySelector: (v: T) => U, keyComparer: Comparer<U>, offset?: number): number { array: readonly T[],
value: T,
keySelector: (v: T) => U,
keyComparer: Comparer<U>,
offset?: number
): number {
return binarySearchKey(array, keySelector(value), keySelector, keyComparer, offset); return binarySearchKey(array, keySelector(value), keySelector, keyComparer, offset);
} }
@ -209,9 +240,13 @@ export function binarySearch<T, U>(array: readonly T[], value: T,
* @param keyComparer A callback used to compare two keys in a sorted array. * @param keyComparer A callback used to compare two keys in a sorted array.
* @param offset An offset into `array` at which to start the search. * @param offset An offset into `array` at which to start the search.
*/ */
export function binarySearchKey<T, U>(array: readonly T[], key: U, export function binarySearchKey<T, U>(
keySelector: (v: T) => U, keyComparer: Comparer<U>, offset?: number): number { array: readonly T[],
key: U,
keySelector: (v: T) => U,
keyComparer: Comparer<U>,
offset?: number
): number {
if (!some(array)) { if (!some(array)) {
return -1; return -1;
} }

18
server/src/common/commandLineOptions.ts
View File

@ -1,13 +1,13 @@
/* /*
* commandLineOptions.ts * commandLineOptions.ts
* Copyright (c) Microsoft Corporation. * Copyright (c) Microsoft Corporation.
* Licensed under the MIT license. * Licensed under the MIT license.
* Author: Eric Traut * Author: Eric Traut
* *
* Class that holds the command-line options (those that can be * Class that holds the command-line options (those that can be
* passed into the main entry point of the command-line version * passed into the main entry point of the command-line version
* of the analyzer). * of the analyzer).
*/ */
// Some options can be specified by command line. // Some options can be specified by command line.
export class CommandLineOptions { export class CommandLineOptions {

319
server/src/common/configOptions.ts
View File

@ -1,18 +1,17 @@
/* /*
* configOptions.ts * configOptions.ts
* Copyright (c) Microsoft Corporation. * Copyright (c) Microsoft Corporation.
* Licensed under the MIT license. * Licensed under the MIT license.
* Author: Eric Traut * Author: Eric Traut
* *
* Class that holds the configuration options for the analyzer. * Class that holds the configuration options for the analyzer.
*/ */
import { isAbsolute } from 'path'; import { isAbsolute } from 'path';
import { ConsoleInterface } from './console'; import { ConsoleInterface } from './console';
import { DiagnosticRule } from './diagnosticRules'; import { DiagnosticRule } from './diagnosticRules';
import { combinePaths, ensureTrailingDirectorySeparator, FileSpec, import { combinePaths, ensureTrailingDirectorySeparator, FileSpec, getFileSpec, normalizePath } from './pathUtils';
getFileSpec, normalizePath } from './pathUtils';
import { latestStablePythonVersion, PythonVersion, versionFromString } from './pythonVersion'; import { latestStablePythonVersion, PythonVersion, versionFromString } from './pythonVersion';
export class ExecutionEnvironment { export class ExecutionEnvironment {
@ -163,9 +162,7 @@ export interface DiagnosticSettings {
reportImplicitStringConcatenation: DiagnosticLevel; reportImplicitStringConcatenation: DiagnosticLevel;
} }
export function cloneDiagnosticSettings( export function cloneDiagnosticSettings(diagSettings: DiagnosticSettings): DiagnosticSettings {
diagSettings: DiagnosticSettings): DiagnosticSettings {
// Create a shallow copy of the existing object. // Create a shallow copy of the existing object.
return Object.assign({}, diagSettings); return Object.assign({}, diagSettings);
} }
@ -399,14 +396,12 @@ export class ConfigOptions {
// execution environment is used. // execution environment is used.
findExecEnvironment(filePath: string): ExecutionEnvironment { findExecEnvironment(filePath: string): ExecutionEnvironment {
let execEnv = this.executionEnvironments.find(env => { let execEnv = this.executionEnvironments.find(env => {
const envRoot = ensureTrailingDirectorySeparator( const envRoot = ensureTrailingDirectorySeparator(normalizePath(combinePaths(this.projectRoot, env.root)));
normalizePath(combinePaths(this.projectRoot, env.root)));
return filePath.startsWith(envRoot); return filePath.startsWith(envRoot);
}); });
if (!execEnv) { if (!execEnv) {
execEnv = new ExecutionEnvironment(this.projectRoot, execEnv = new ExecutionEnvironment(this.projectRoot, this.defaultPythonVersion, this.defaultPythonPlatform);
this.defaultPythonVersion, this.defaultPythonPlatform);
} }
return execEnv; return execEnv;
@ -423,9 +418,9 @@ export class ConfigOptions {
const filesList = configObj.include as string[]; const filesList = configObj.include as string[];
filesList.forEach((fileSpec, index) => { filesList.forEach((fileSpec, index) => {
if (typeof fileSpec !== 'string') { if (typeof fileSpec !== 'string') {
console.log(`Index ${ index } of "include" array should be a string.`); console.log(`Index ${index} of "include" array should be a string.`);
} else if (isAbsolute(fileSpec)) { } else if (isAbsolute(fileSpec)) {
console.log(`Ignoring path "${ fileSpec }" in "include" array because it is not relative.`); console.log(`Ignoring path "${fileSpec}" in "include" array because it is not relative.`);
} else { } else {
this.include.push(getFileSpec(this.projectRoot, fileSpec)); this.include.push(getFileSpec(this.projectRoot, fileSpec));
} }
@ -442,9 +437,9 @@ export class ConfigOptions {
const filesList = configObj.exclude as string[]; const filesList = configObj.exclude as string[];
filesList.forEach((fileSpec, index) => { filesList.forEach((fileSpec, index) => {
if (typeof fileSpec !== 'string') { if (typeof fileSpec !== 'string') {
console.log(`Index ${ index } of "exclude" array should be a string.`); console.log(`Index ${index} of "exclude" array should be a string.`);
} else if (isAbsolute(fileSpec)) { } else if (isAbsolute(fileSpec)) {
console.log(`Ignoring path "${ fileSpec }" in "exclude" array because it is not relative.`); console.log(`Ignoring path "${fileSpec}" in "exclude" array because it is not relative.`);
} else { } else {
this.exclude.push(getFileSpec(this.projectRoot, fileSpec)); this.exclude.push(getFileSpec(this.projectRoot, fileSpec));
} }
@ -461,9 +456,9 @@ export class ConfigOptions {
const filesList = configObj.ignore as string[]; const filesList = configObj.ignore as string[];
filesList.forEach((fileSpec, index) => { filesList.forEach((fileSpec, index) => {
if (typeof fileSpec !== 'string') { if (typeof fileSpec !== 'string') {
console.log(`Index ${ index } of "ignore" array should be a string.`); console.log(`Index ${index} of "ignore" array should be a string.`);
} else if (isAbsolute(fileSpec)) { } else if (isAbsolute(fileSpec)) {
console.log(`Ignoring path "${ fileSpec }" in "ignore" array because it is not relative.`); console.log(`Ignoring path "${fileSpec}" in "ignore" array because it is not relative.`);
} else { } else {
this.ignore.push(getFileSpec(this.projectRoot, fileSpec)); this.ignore.push(getFileSpec(this.projectRoot, fileSpec));
} }
@ -480,9 +475,9 @@ export class ConfigOptions {
const filesList = configObj.strict as string[]; const filesList = configObj.strict as string[];
filesList.forEach((fileSpec, index) => { filesList.forEach((fileSpec, index) => {
if (typeof fileSpec !== 'string') { if (typeof fileSpec !== 'string') {
console.log(`Index ${ index } of "strict" array should be a string.`); console.log(`Index ${index} of "strict" array should be a string.`);
} else if (isAbsolute(fileSpec)) { } else if (isAbsolute(fileSpec)) {
console.log(`Ignoring path "${ fileSpec }" in "strict" array because it is not relative.`); console.log(`Ignoring path "${fileSpec}" in "strict" array because it is not relative.`);
} else { } else {
this.strict.push(getFileSpec(this.projectRoot, fileSpec)); this.strict.push(getFileSpec(this.projectRoot, fileSpec));
} }
@ -495,194 +490,270 @@ export class ConfigOptions {
this.diagnosticSettings = { this.diagnosticSettings = {
// Use strict inference rules for list expressions? // Use strict inference rules for list expressions?
strictListInference: this._convertBoolean( strictListInference: this._convertBoolean(
configObj.strictListInference, DiagnosticRule.strictListInference, configObj.strictListInference,
defaultSettings.strictListInference), DiagnosticRule.strictListInference,
defaultSettings.strictListInference
),
// Use strict inference rules for dictionary expressions? // Use strict inference rules for dictionary expressions?
strictDictionaryInference: this._convertBoolean( strictDictionaryInference: this._convertBoolean(
configObj.strictDictionaryInference, DiagnosticRule.strictDictionaryInference, configObj.strictDictionaryInference,
defaultSettings.strictDictionaryInference), DiagnosticRule.strictDictionaryInference,
defaultSettings.strictDictionaryInference
),
// Should a None default value imply that the parameter type // Should a None default value imply that the parameter type
// is Optional? // is Optional?
strictParameterNoneValue: this._convertBoolean( strictParameterNoneValue: this._convertBoolean(
configObj.strictParameterNoneValue, DiagnosticRule.strictParameterNoneValue, configObj.strictParameterNoneValue,
defaultSettings.strictParameterNoneValue), DiagnosticRule.strictParameterNoneValue,
defaultSettings.strictParameterNoneValue
),
// Should "# type: ignore" be honored? // Should "# type: ignore" be honored?
enableTypeIgnoreComments: this._convertBoolean( enableTypeIgnoreComments: this._convertBoolean(
configObj.enableTypeIgnoreComments, DiagnosticRule.enableTypeIgnoreComments, configObj.enableTypeIgnoreComments,
defaultSettings.enableTypeIgnoreComments), DiagnosticRule.enableTypeIgnoreComments,
defaultSettings.enableTypeIgnoreComments
),
// Read the "reportTypeshedErrors" entry. // Read the "reportTypeshedErrors" entry.
reportTypeshedErrors: this._convertDiagnosticLevel( reportTypeshedErrors: this._convertDiagnosticLevel(
configObj.reportTypeshedErrors, DiagnosticRule.reportTypeshedErrors, configObj.reportTypeshedErrors,
defaultSettings.reportTypeshedErrors), DiagnosticRule.reportTypeshedErrors,
defaultSettings.reportTypeshedErrors
),
// Read the "reportMissingImports" entry. // Read the "reportMissingImports" entry.
reportMissingImports: this._convertDiagnosticLevel( reportMissingImports: this._convertDiagnosticLevel(
configObj.reportMissingImports, DiagnosticRule.reportMissingImports, configObj.reportMissingImports,
defaultSettings.reportMissingImports), DiagnosticRule.reportMissingImports,
defaultSettings.reportMissingImports
),
// Read the "reportUnusedImport" entry. // Read the "reportUnusedImport" entry.
reportUnusedImport: this._convertDiagnosticLevel( reportUnusedImport: this._convertDiagnosticLevel(
configObj.reportUnusedImport, DiagnosticRule.reportUnusedImport, configObj.reportUnusedImport,
defaultSettings.reportUnusedImport), DiagnosticRule.reportUnusedImport,
defaultSettings.reportUnusedImport
),
// Read the "reportUnusedClass" entry. // Read the "reportUnusedClass" entry.
reportUnusedClass: this._convertDiagnosticLevel( reportUnusedClass: this._convertDiagnosticLevel(
configObj.reportUnusedClass, DiagnosticRule.reportUnusedClass, configObj.reportUnusedClass,
defaultSettings.reportUnusedClass), DiagnosticRule.reportUnusedClass,
defaultSettings.reportUnusedClass
),
// Read the "reportUnusedFunction" entry. // Read the "reportUnusedFunction" entry.
reportUnusedFunction: this._convertDiagnosticLevel( reportUnusedFunction: this._convertDiagnosticLevel(
configObj.reportUnusedFunction, DiagnosticRule.reportUnusedFunction, configObj.reportUnusedFunction,
defaultSettings.reportUnusedFunction), DiagnosticRule.reportUnusedFunction,
defaultSettings.reportUnusedFunction
),
// Read the "reportUnusedVariable" entry. // Read the "reportUnusedVariable" entry.
reportUnusedVariable: this._convertDiagnosticLevel( reportUnusedVariable: this._convertDiagnosticLevel(
configObj.reportUnusedVariable, DiagnosticRule.reportUnusedVariable, configObj.reportUnusedVariable,
defaultSettings.reportUnusedVariable), DiagnosticRule.reportUnusedVariable,
defaultSettings.reportUnusedVariable
),
// Read the "reportDuplicateImport" entry. // Read the "reportDuplicateImport" entry.
reportDuplicateImport: this._convertDiagnosticLevel( reportDuplicateImport: this._convertDiagnosticLevel(
configObj.reportDuplicateImport, DiagnosticRule.reportDuplicateImport, configObj.reportDuplicateImport,
defaultSettings.reportDuplicateImport), DiagnosticRule.reportDuplicateImport,
defaultSettings.reportDuplicateImport
),
// Read the "reportMissingTypeStubs" entry. // Read the "reportMissingTypeStubs" entry.
reportMissingTypeStubs: this._convertDiagnosticLevel( reportMissingTypeStubs: this._convertDiagnosticLevel(
configObj.reportMissingTypeStubs, DiagnosticRule.reportMissingTypeStubs, configObj.reportMissingTypeStubs,
defaultSettings.reportMissingTypeStubs), DiagnosticRule.reportMissingTypeStubs,
defaultSettings.reportMissingTypeStubs
),
// Read the "reportImportCycles" entry. // Read the "reportImportCycles" entry.
reportImportCycles: this._convertDiagnosticLevel( reportImportCycles: this._convertDiagnosticLevel(
configObj.reportImportCycles, DiagnosticRule.reportImportCycles, configObj.reportImportCycles,
defaultSettings.reportImportCycles), DiagnosticRule.reportImportCycles,
defaultSettings.reportImportCycles
),
// Read the "reportOptionalSubscript" entry. // Read the "reportOptionalSubscript" entry.
reportOptionalSubscript: this._convertDiagnosticLevel( reportOptionalSubscript: this._convertDiagnosticLevel(
configObj.reportOptionalSubscript, DiagnosticRule.reportOptionalSubscript, configObj.reportOptionalSubscript,
defaultSettings.reportOptionalSubscript), DiagnosticRule.reportOptionalSubscript,
defaultSettings.reportOptionalSubscript
),
// Read the "reportOptionalMemberAccess" entry. // Read the "reportOptionalMemberAccess" entry.
reportOptionalMemberAccess: this._convertDiagnosticLevel( reportOptionalMemberAccess: this._convertDiagnosticLevel(
configObj.reportOptionalMemberAccess, DiagnosticRule.reportOptionalMemberAccess, configObj.reportOptionalMemberAccess,
defaultSettings.reportOptionalMemberAccess), DiagnosticRule.reportOptionalMemberAccess,
defaultSettings.reportOptionalMemberAccess
),
// Read the "reportOptionalCall" entry. // Read the "reportOptionalCall" entry.
reportOptionalCall: this._convertDiagnosticLevel( reportOptionalCall: this._convertDiagnosticLevel(
configObj.reportOptionalCall, DiagnosticRule.reportOptionalCall, configObj.reportOptionalCall,
defaultSettings.reportOptionalCall), DiagnosticRule.reportOptionalCall,
defaultSettings.reportOptionalCall
),
// Read the "reportOptionalIterable" entry. // Read the "reportOptionalIterable" entry.
reportOptionalIterable: this._convertDiagnosticLevel( reportOptionalIterable: this._convertDiagnosticLevel(
configObj.reportOptionalIterable, DiagnosticRule.reportOptionalIterable, configObj.reportOptionalIterable,
defaultSettings.reportOptionalIterable), DiagnosticRule.reportOptionalIterable,
defaultSettings.reportOptionalIterable
),
// Read the "reportOptionalContextManager" entry. // Read the "reportOptionalContextManager" entry.
reportOptionalContextManager: this._convertDiagnosticLevel( reportOptionalContextManager: this._convertDiagnosticLevel(
configObj.reportOptionalContextManager, DiagnosticRule.reportOptionalContextManager, configObj.reportOptionalContextManager,
defaultSettings.reportOptionalContextManager), DiagnosticRule.reportOptionalContextManager,
defaultSettings.reportOptionalContextManager
),
// Read the "reportOptionalOperand" entry. // Read the "reportOptionalOperand" entry.
reportOptionalOperand: this._convertDiagnosticLevel( reportOptionalOperand: this._convertDiagnosticLevel(
configObj.reportOptionalOperand, DiagnosticRule.reportOptionalOperand, configObj.reportOptionalOperand,
defaultSettings.reportOptionalOperand), DiagnosticRule.reportOptionalOperand,
defaultSettings.reportOptionalOperand
),
// Read the "reportUntypedFunctionDecorator" entry. // Read the "reportUntypedFunctionDecorator" entry.
reportUntypedFunctionDecorator: this._convertDiagnosticLevel( reportUntypedFunctionDecorator: this._convertDiagnosticLevel(
configObj.reportUntypedFunctionDecorator, DiagnosticRule.reportUntypedFunctionDecorator, configObj.reportUntypedFunctionDecorator,
defaultSettings.reportUntypedFunctionDecorator), DiagnosticRule.reportUntypedFunctionDecorator,
defaultSettings.reportUntypedFunctionDecorator
),
// Read the "reportUntypedClassDecorator" entry. // Read the "reportUntypedClassDecorator" entry.
reportUntypedClassDecorator: this._convertDiagnosticLevel( reportUntypedClassDecorator: this._convertDiagnosticLevel(
configObj.reportUntypedClassDecorator, DiagnosticRule.reportUntypedClassDecorator, configObj.reportUntypedClassDecorator,
defaultSettings.reportUntypedClassDecorator), DiagnosticRule.reportUntypedClassDecorator,
defaultSettings.reportUntypedClassDecorator
),
// Read the "reportUntypedBaseClass" entry. // Read the "reportUntypedBaseClass" entry.
reportUntypedBaseClass: this._convertDiagnosticLevel( reportUntypedBaseClass: this._convertDiagnosticLevel(
configObj.reportUntypedBaseClass, DiagnosticRule.reportUntypedBaseClass, configObj.reportUntypedBaseClass,
defaultSettings.reportUntypedBaseClass), DiagnosticRule.reportUntypedBaseClass,
defaultSettings.reportUntypedBaseClass
),
// Read the "reportUntypedNamedTuple" entry. // Read the "reportUntypedNamedTuple" entry.
reportUntypedNamedTuple: this._convertDiagnosticLevel( reportUntypedNamedTuple: this._convertDiagnosticLevel(
configObj.reportUntypedNamedTuple, DiagnosticRule.reportUntypedNamedTuple, configObj.reportUntypedNamedTuple,
defaultSettings.reportUntypedNamedTuple), DiagnosticRule.reportUntypedNamedTuple,
defaultSettings.reportUntypedNamedTuple
),
// Read the "reportPrivateUsage" entry. // Read the "reportPrivateUsage" entry.
reportPrivateUsage: this._convertDiagnosticLevel( reportPrivateUsage: this._convertDiagnosticLevel(
configObj.reportPrivateUsage, DiagnosticRule.reportPrivateUsage, configObj.reportPrivateUsage,
defaultSettings.reportPrivateUsage), DiagnosticRule.reportPrivateUsage,
defaultSettings.reportPrivateUsage
),
// Read the "reportConstantRedefinition" entry. // Read the "reportConstantRedefinition" entry.
reportConstantRedefinition: this._convertDiagnosticLevel( reportConstantRedefinition: this._convertDiagnosticLevel(
configObj.reportConstantRedefinition, DiagnosticRule.reportConstantRedefinition, configObj.reportConstantRedefinition,
defaultSettings.reportConstantRedefinition), DiagnosticRule.reportConstantRedefinition,
defaultSettings.reportConstantRedefinition
),
// Read the "reportIncompatibleMethodOverride" entry. // Read the "reportIncompatibleMethodOverride" entry.
reportIncompatibleMethodOverride: this._convertDiagnosticLevel( reportIncompatibleMethodOverride: this._convertDiagnosticLevel(
configObj.reportIncompatibleMethodOverride, DiagnosticRule.reportIncompatibleMethodOverride, configObj.reportIncompatibleMethodOverride,
defaultSettings.reportIncompatibleMethodOverride), DiagnosticRule.reportIncompatibleMethodOverride,
defaultSettings.reportIncompatibleMethodOverride
),
// Read the "reportInvalidStringEscapeSequence" entry. // Read the "reportInvalidStringEscapeSequence" entry.
reportInvalidStringEscapeSequence: this._convertDiagnosticLevel( reportInvalidStringEscapeSequence: this._convertDiagnosticLevel(
configObj.reportInvalidStringEscapeSequence, DiagnosticRule.reportInvalidStringEscapeSequence, configObj.reportInvalidStringEscapeSequence,
defaultSettings.reportInvalidStringEscapeSequence), DiagnosticRule.reportInvalidStringEscapeSequence,
defaultSettings.reportInvalidStringEscapeSequence
),
// Read the "reportUnknownParameterType" entry. // Read the "reportUnknownParameterType" entry.
reportUnknownParameterType: this._convertDiagnosticLevel( reportUnknownParameterType: this._convertDiagnosticLevel(
configObj.reportUnknownParameterType, DiagnosticRule.reportUnknownParameterType, configObj.reportUnknownParameterType,
defaultSettings.reportUnknownParameterType), DiagnosticRule.reportUnknownParameterType,
defaultSettings.reportUnknownParameterType
),
// Read the "reportUnknownArgumentType" entry. // Read the "reportUnknownArgumentType" entry.
reportUnknownArgumentType: this._convertDiagnosticLevel( reportUnknownArgumentType: this._convertDiagnosticLevel(
configObj.reportUnknownArgumentType, DiagnosticRule.reportUnknownArgumentType, configObj.reportUnknownArgumentType,
defaultSettings.reportUnknownArgumentType), DiagnosticRule.reportUnknownArgumentType,
defaultSettings.reportUnknownArgumentType
),
// Read the "reportUnknownLambdaType" entry. // Read the "reportUnknownLambdaType" entry.
reportUnknownLambdaType: this._convertDiagnosticLevel( reportUnknownLambdaType: this._convertDiagnosticLevel(
configObj.reportUnknownLambdaType, DiagnosticRule.reportUnknownLambdaType, configObj.reportUnknownLambdaType,
defaultSettings.reportUnknownLambdaType), DiagnosticRule.reportUnknownLambdaType,
defaultSettings.reportUnknownLambdaType
),
// Read the "reportUnknownVariableType" entry. // Read the "reportUnknownVariableType" entry.
reportUnknownVariableType: this._convertDiagnosticLevel( reportUnknownVariableType: this._convertDiagnosticLevel(
configObj.reportUnknownVariableType, DiagnosticRule.reportUnknownVariableType, configObj.reportUnknownVariableType,
defaultSettings.reportUnknownVariableType), DiagnosticRule.reportUnknownVariableType,
defaultSettings.reportUnknownVariableType
),
// Read the "reportUnknownMemberType" entry. // Read the "reportUnknownMemberType" entry.
reportUnknownMemberType: this._convertDiagnosticLevel( reportUnknownMemberType: this._convertDiagnosticLevel(
configObj.reportUnknownMemberType, DiagnosticRule.reportUnknownMemberType, configObj.reportUnknownMemberType,
defaultSettings.reportUnknownMemberType), DiagnosticRule.reportUnknownMemberType,
defaultSettings.reportUnknownMemberType
),
// Read the "reportCallInDefaultInitializer" entry. // Read the "reportCallInDefaultInitializer" entry.
reportCallInDefaultInitializer: this._convertDiagnosticLevel( reportCallInDefaultInitializer: this._convertDiagnosticLevel(
configObj.reportCallInDefaultInitializer, DiagnosticRule.reportCallInDefaultInitializer, configObj.reportCallInDefaultInitializer,
defaultSettings.reportCallInDefaultInitializer), DiagnosticRule.reportCallInDefaultInitializer,
defaultSettings.reportCallInDefaultInitializer
),
// Read the "reportUnnecessaryIsInstance" entry. // Read the "reportUnnecessaryIsInstance" entry.
reportUnnecessaryIsInstance: this._convertDiagnosticLevel( reportUnnecessaryIsInstance: this._convertDiagnosticLevel(
configObj.reportUnnecessaryIsInstance, DiagnosticRule.reportUnnecessaryIsInstance, configObj.reportUnnecessaryIsInstance,
defaultSettings.reportUnnecessaryIsInstance), DiagnosticRule.reportUnnecessaryIsInstance,
defaultSettings.reportUnnecessaryIsInstance
),
// Read the "reportUnnecessaryCast" entry. // Read the "reportUnnecessaryCast" entry.
reportUnnecessaryCast: this._convertDiagnosticLevel( reportUnnecessaryCast: this._convertDiagnosticLevel(
configObj.reportUnnecessaryCast, DiagnosticRule.reportUnnecessaryCast, configObj.reportUnnecessaryCast,
defaultSettings.reportUnnecessaryCast), DiagnosticRule.reportUnnecessaryCast,
defaultSettings.reportUnnecessaryCast
),
// Read the "reportAssertAlwaysTrue" entry. // Read the "reportAssertAlwaysTrue" entry.
reportAssertAlwaysTrue: this._convertDiagnosticLevel( reportAssertAlwaysTrue: this._convertDiagnosticLevel(
configObj.reportAssertAlwaysTrue, DiagnosticRule.reportAssertAlwaysTrue, configObj.reportAssertAlwaysTrue,
defaultSettings.reportAssertAlwaysTrue), DiagnosticRule.reportAssertAlwaysTrue,
defaultSettings.reportAssertAlwaysTrue
),
// Read the "reportSelfClsParameterName" entry. // Read the "reportSelfClsParameterName" entry.
reportSelfClsParameterName: this._convertDiagnosticLevel( reportSelfClsParameterName: this._convertDiagnosticLevel(
configObj.reportSelfClsParameterName, DiagnosticRule.reportSelfClsParameterName, configObj.reportSelfClsParameterName,
defaultSettings.reportSelfClsParameterName), DiagnosticRule.reportSelfClsParameterName,
defaultSettings.reportSelfClsParameterName
),
// Read the "reportImplicitStringConcatenation" entry. // Read the "reportImplicitStringConcatenation" entry.
reportImplicitStringConcatenation: this._convertDiagnosticLevel( reportImplicitStringConcatenation: this._convertDiagnosticLevel(
configObj.reportImplicitStringConcatenation, DiagnosticRule.reportImplicitStringConcatenation, configObj.reportImplicitStringConcatenation,
defaultSettings.reportImplicitStringConcatenation) DiagnosticRule.reportImplicitStringConcatenation,
defaultSettings.reportImplicitStringConcatenation
)
}; };
// Read the "venvPath". // Read the "venvPath".
@ -736,9 +807,9 @@ export class ConfigOptions {
if (typeof configObj.typeshedPath !== 'string') { if (typeof configObj.typeshedPath !== 'string') {
console.log(`Config "typeshedPath" field must contain a string.`); console.log(`Config "typeshedPath" field must contain a string.`);
} else { } else {
this.typeshedPath = configObj.typeshedPath ? this.typeshedPath = configObj.typeshedPath
normalizePath(combinePaths(this.projectRoot, configObj.typeshedPath)) : ? normalizePath(combinePaths(this.projectRoot, configObj.typeshedPath))
''; : '';
} }
} }
@ -777,13 +848,11 @@ export class ConfigOptions {
return value ? true : false; return value ? true : false;
} }
console.log(`Config "${ fieldName }" entry must be true or false.`); console.log(`Config "${fieldName}" entry must be true or false.`);
return defaultValue; return defaultValue;
} }
private _convertDiagnosticLevel(value: any, fieldName: string, private _convertDiagnosticLevel(value: any, fieldName: string, defaultValue: DiagnosticLevel): DiagnosticLevel {
defaultValue: DiagnosticLevel): DiagnosticLevel {
if (value === undefined) { if (value === undefined) {
return defaultValue; return defaultValue;
} else if (typeof value === 'boolean') { } else if (typeof value === 'boolean') {
@ -794,33 +863,41 @@ export class ConfigOptions {
} }
} }
console.log(`Config "${ fieldName }" entry must be true, false, "error", "warning" or "none".`); console.log(`Config "${fieldName}" entry must be true, false, "error", "warning" or "none".`);
return defaultValue; return defaultValue;
} }
private _initExecutionEnvironmentFromJson(envObj: any, index: number, private _initExecutionEnvironmentFromJson(
console: ConsoleInterface): ExecutionEnvironment | undefined { envObj: any,
index: number,
console: ConsoleInterface
): ExecutionEnvironment | undefined {
try { try {
const newExecEnv = new ExecutionEnvironment(this.projectRoot, const newExecEnv = new ExecutionEnvironment(
this.defaultPythonVersion, this.defaultPythonPlatform); this.projectRoot,
this.defaultPythonVersion,
this.defaultPythonPlatform
);
// Validate the root. // Validate the root.
if (envObj.root && typeof envObj.root === 'string') { if (envObj.root && typeof envObj.root === 'string') {
newExecEnv.root = normalizePath(combinePaths(this.projectRoot, envObj.root)); newExecEnv.root = normalizePath(combinePaths(this.projectRoot, envObj.root));
} else { } else {
console.log(`Config executionEnvironments index ${ index }: missing root value.`); console.log(`Config executionEnvironments index ${index}: missing root value.`);
} }
// Validate the extraPaths. // Validate the extraPaths.
if (envObj.extraPaths) { if (envObj.extraPaths) {
if (!Array.isArray(envObj.extraPaths)) { if (!Array.isArray(envObj.extraPaths)) {
console.log(`Config executionEnvironments index ${ index }: extraPaths field must contain an array.`); console.log(`Config executionEnvironments index ${index}: extraPaths field must contain an array.`);
} else { } else {
const pathList = envObj.extraPaths as string[]; const pathList = envObj.extraPaths as string[];
pathList.forEach((path, pathIndex) => { pathList.forEach((path, pathIndex) => {
if (typeof path !== 'string') { if (typeof path !== 'string') {
console.log(`Config executionEnvironments index ${ index }:` + console.log(
` extraPaths field ${ pathIndex } must be a string.`); `Config executionEnvironments index ${index}:` +
` extraPaths field ${pathIndex} must be a string.`
);
} else { } else {
newExecEnv.extraPaths.push(normalizePath(combinePaths(this.projectRoot, path))); newExecEnv.extraPaths.push(normalizePath(combinePaths(this.projectRoot, path)));
} }
@ -835,10 +912,10 @@ export class ConfigOptions {
if (version) { if (version) {
newExecEnv.pythonVersion = version; newExecEnv.pythonVersion = version;
} else { } else {
console.log(`Config executionEnvironments index ${ index } contains unsupported pythonVersion.`); console.log(`Config executionEnvironments index ${index} contains unsupported pythonVersion.`);
} }
} else { } else {
console.log(`Config executionEnvironments index ${ index } pythonVersion must be a string.`); console.log(`Config executionEnvironments index ${index} pythonVersion must be a string.`);
} }
} }
@ -847,7 +924,7 @@ export class ConfigOptions {
if (typeof envObj.pythonPlatform === 'string') { if (typeof envObj.pythonPlatform === 'string') {
newExecEnv.pythonPlatform = envObj.pythonPlatform; newExecEnv.pythonPlatform = envObj.pythonPlatform;
} else { } else {
console.log(`Config executionEnvironments index ${ index } pythonPlatform must be a string.`); console.log(`Config executionEnvironments index ${index} pythonPlatform must be a string.`);
} }
} }
@ -856,13 +933,13 @@ export class ConfigOptions {
if (typeof envObj.venv === 'string') { if (typeof envObj.venv === 'string') {
newExecEnv.venv = envObj.venv; newExecEnv.venv = envObj.venv;
} else { } else {
console.log(`Config executionEnvironments index ${ index } venv must be a string.`); console.log(`Config executionEnvironments index ${index} venv must be a string.`);
} }
} }
return newExecEnv; return newExecEnv;
} catch { } catch {
console.log(`Config executionEnvironments index ${ index } is not accessible.`); console.log(`Config executionEnvironments index ${index} is not accessible.`);
} }
return undefined; return undefined;

16
server/src/common/console.ts
View File

@ -1,12 +1,12 @@
/* /*
* console.ts * console.ts
* Copyright (c) Microsoft Corporation. * Copyright (c) Microsoft Corporation.
* Licensed under the MIT license. * Licensed under the MIT license.
* Author: Eric Traut * Author: Eric Traut
* *
* Provides an abstraction for console logging and error-reporting * Provides an abstraction for console logging and error-reporting
* methods. * methods.
*/ */
export interface ConsoleInterface { export interface ConsoleInterface {
log: (message: string) => void; log: (message: string) => void;

38
server/src/common/core.ts
View File

@ -21,32 +21,48 @@ export const enum Comparison {
export type AnyFunction = (...args: never[]) => void; export type AnyFunction = (...args: never[]) => void;
/** Do nothing and return false */ /** Do nothing and return false */
export function returnFalse(): false { return false; } export function returnFalse(): false {
return false;
}
/** Do nothing and return true */ /** Do nothing and return true */
export function returnTrue(): true { return true; } export function returnTrue(): true {
return true;
}
/** Do nothing and return undefined */ /** Do nothing and return undefined */
export function returnUndefined(): undefined { return undefined; } export function returnUndefined(): undefined {
return undefined;
}
/** Returns its argument. */ /** Returns its argument. */
export function identity<T>(x: T) { return x; } export function identity<T>(x: T) {
return x;
}
/** Returns lower case string */ /** Returns lower case string */
export function toLowerCase(x: string) { return x.toLowerCase(); } export function toLowerCase(x: string) {
return x.toLowerCase();
}
export function equateValues<T>(a: T, b: T) { return a === b; } export function equateValues<T>(a: T, b: T) {
return a === b;
}
export type GetCanonicalFileName = (fileName: string) => string; export type GetCanonicalFileName = (fileName: string) => string;
export function compareComparableValues(a: string | undefined, b: string | undefined): Comparison; export function compareComparableValues(a: string | undefined, b: string | undefined): Comparison;
export function compareComparableValues(a: number | undefined, b: number | undefined): Comparison; export function compareComparableValues(a: number | undefined, b: number | undefined): Comparison;
export function compareComparableValues(a: string | number | undefined, b: string | number | undefined) { export function compareComparableValues(a: string | number | undefined, b: string | number | undefined) {
return a === b ? Comparison.EqualTo : return a === b
a === undefined ? Comparison.LessThan : ? Comparison.EqualTo
b === undefined ? Comparison.GreaterThan : : a === undefined
a < b ? Comparison.LessThan : ? Comparison.LessThan
Comparison.GreaterThan; : b === undefined
? Comparison.GreaterThan
: a < b
? Comparison.LessThan
: Comparison.GreaterThan;
} }
/** /**

25
server/src/common/debug.ts
View File

@ -4,17 +4,22 @@
* Licensed under the MIT license. * Licensed under the MIT license.
* *
* Various debug helper methods to show user friendly debugging info * Various debug helper methods to show user friendly debugging info
*/ */
import { stableSort } from './collectionUtils'; import { stableSort } from './collectionUtils';
import { AnyFunction, compareValues, hasProperty } from './core'; import { AnyFunction, compareValues, hasProperty } from './core';
export function assert(expression: boolean, message?: string, export function assert(
verboseDebugInfo?: string | (() => string), stackCrawlMark?: AnyFunction): void { expression: boolean,
message?: string,
verboseDebugInfo?: string | (() => string),
stackCrawlMark?: AnyFunction
): void {
if (!expression) { if (!expression) {
if (verboseDebugInfo) { if (verboseDebugInfo) {
message += '\r\nVerbose Debug Information: ' + (typeof verboseDebugInfo === 'string' ? verboseDebugInfo : verboseDebugInfo()); message +=
'\r\nVerbose Debug Information: ' +
(typeof verboseDebugInfo === 'string' ? verboseDebugInfo : verboseDebugInfo());
} }
fail(message ? 'False expression: ' + message : 'False expression.', stackCrawlMark || assert); fail(message ? 'False expression: ' + message : 'False expression.', stackCrawlMark || assert);
} }
@ -22,7 +27,7 @@ export function assert(expression: boolean, message?: string,
export function fail(message?: string, stackCrawlMark?: AnyFunction): never { export function fail(message?: string, stackCrawlMark?: AnyFunction): never {
// debugger; // debugger;
const e = new Error(message ? `Debug Failure. ${ message }` : 'Debug Failure.'); const e = new Error(message ? `Debug Failure. ${message}` : 'Debug Failure.');
if ((Error as any).captureStackTrace) { if ((Error as any).captureStackTrace) {
(Error as any).captureStackTrace(e, stackCrawlMark || fail); (Error as any).captureStackTrace(e, stackCrawlMark || fail);
} }
@ -30,7 +35,9 @@ export function fail(message?: string, stackCrawlMark?: AnyFunction): never {
} }
export function assertDefined<T>(value: T | null | undefined, message?: string): T { export function assertDefined<T>(value: T | null | undefined, message?: string): T {
if (value === undefined || value === null) { return fail(message); } if (value === undefined || value === null) {
return fail(message);
}
return value; return value;
} }
@ -43,7 +50,7 @@ export function assertEachDefined<T, A extends readonly T[]>(value: A, message?:
export function assertNever(member: never, message = 'Illegal value:', stackCrawlMark?: AnyFunction): never { export function assertNever(member: never, message = 'Illegal value:', stackCrawlMark?: AnyFunction): never {
const detail = JSON.stringify(member); const detail = JSON.stringify(member);
return fail(`${ message } ${ detail }`, stackCrawlMark || assertNever); return fail(`${message} ${detail}`, stackCrawlMark || assertNever);
} }
export function getFunctionName(func: AnyFunction) { export function getFunctionName(func: AnyFunction) {
@ -74,7 +81,7 @@ export function formatEnum(value = 0, enumObject: any, isFlags?: boolean) {
break; break;
} }
if (enumValue !== 0 && enumValue & value) { if (enumValue !== 0 && enumValue & value) {
result = `${ result }${ result ? '|' : '' }${ enumName }`; result = `${result}${result ? '|' : ''}${enumName}`;
remainingFlags &= ~enumValue; remainingFlags &= ~enumValue;
} }
} }

18
server/src/common/diagnostic.ts
View File

@ -1,11 +1,11 @@
/* /*
* diagnostics.ts * diagnostics.ts
* Copyright (c) Microsoft Corporation. * Copyright (c) Microsoft Corporation.
* Licensed under the MIT license. * Licensed under the MIT license.
* Author: Eric Traut * Author: Eric Traut
* *
* Class that represents errors and warnings. * Class that represents errors and warnings.
*/ */
import { Commands } from '../commands/commands'; import { Commands } from '../commands/commands';
import { Range } from './textRange'; import { Range } from './textRange';
@ -42,9 +42,7 @@ export class Diagnostic {
private _rule: string | undefined; private _rule: string | undefined;
private _relatedInfo: DiagnosticRelatedInfo[] = []; private _relatedInfo: DiagnosticRelatedInfo[] = [];
constructor(readonly category: DiagnosticCategory, readonly message: string, constructor(readonly category: DiagnosticCategory, readonly message: string, readonly range: Range) {}
readonly range: Range) {
}
addAction(action: DiagnosticAction) { addAction(action: DiagnosticAction) {
if (this._actions === undefined) { if (this._actions === undefined) {

16
server/src/common/diagnosticRules.ts
View File

@ -1,12 +1,12 @@
/* /*
* diagnosticRules.ts * diagnosticRules.ts
* Copyright (c) Microsoft Corporation. * Copyright (c) Microsoft Corporation.
* Licensed under the MIT license. * Licensed under the MIT license.
* Author: Eric Traut * Author: Eric Traut
* *
* Strings that represent each of the diagnostic rules * Strings that represent each of the diagnostic rules
* that can be enabled or disabled in the configuration. * that can be enabled or disabled in the configuration.
*/ */
export const enum DiagnosticRule { export const enum DiagnosticRule {
strictListInference = 'strictListInference', strictListInference = 'strictListInference',

16
server/src/common/diagnosticSink.ts
View File

@ -1,15 +1,15 @@
/* /*
* diagnostics.ts * diagnostics.ts
* Copyright (c) Microsoft Corporation. * Copyright (c) Microsoft Corporation.
* Licensed under the MIT license. * Licensed under the MIT license.
* Author: Eric Traut * Author: Eric Traut
* *
* Class that represents errors and warnings. * Class that represents errors and warnings.
*/ */
import { Diagnostic, DiagnosticCategory } from './diagnostic'; import { Diagnostic, DiagnosticCategory } from './diagnostic';
import { convertOffsetsToRange } from './positionUtils'; import { convertOffsetsToRange } from './positionUtils';
import { Range,TextRange } from './textRange'; import { Range, TextRange } from './textRange';
import { TextRangeCollection } from './textRangeCollection'; import { TextRangeCollection } from './textRangeCollection';
// Represents a collection of diagnostics within a file. // Represents a collection of diagnostics within a file.

16
server/src/common/editAction.ts
View File

@ -1,13 +1,13 @@
/* /*
* editAction.ts * editAction.ts
* Copyright (c) Microsoft Corporation. * Copyright (c) Microsoft Corporation.
* Licensed under the MIT license. * Licensed under the MIT license.
* Author: Eric Traut * Author: Eric Traut
* *
* Represents a single edit within a file. * Represents a single edit within a file.
*/ */
import { Range } from "./textRange"; import { Range } from './textRange';
export interface TextEditAction { export interface TextEditAction {
range: Range; range: Range;

18
server/src/common/extensions.ts
View File

@ -1,15 +1,15 @@
/* /*
* extensions.ts * extensions.ts
* Copyright (c) Microsoft Corporation. * Copyright (c) Microsoft Corporation.
* Licensed under the MIT license. * Licensed under the MIT license.
* Author: Eric Traut * Author: Eric Traut
* *
* Extension methods to various types. * Extension methods to various types.
*/ */
/* eslint-disable @typescript-eslint/no-empty-function */ /* eslint-disable @typescript-eslint/no-empty-function */
// Explicitly tells that promise should be run asynchronously. // Explicitly tells that promise should be run asynchronously.
Promise.prototype.ignoreErrors = function <T>(this: Promise<T>) { Promise.prototype.ignoreErrors = function<T>(this: Promise<T>) {
this.catch(() => { }); this.catch(() => {});
}; };

232
server/src/common/pathUtils.ts
View File

@ -1,11 +1,11 @@
/* /*
* pathUtils.ts * pathUtils.ts
* Copyright (c) Microsoft Corporation. * Copyright (c) Microsoft Corporation.
* Licensed under the MIT license. * Licensed under the MIT license.
* Author: Eric Traut * Author: Eric Traut
* *
* Pathname utility functions. * Pathname utility functions.
*/ */
import * as path from 'path'; import * as path from 'path';
import Char from 'typescript-char'; import Char from 'typescript-char';
@ -15,8 +15,11 @@ import { some } from './collectionUtils';
import { compareValues, Comparison, GetCanonicalFileName, identity } from './core'; import { compareValues, Comparison, GetCanonicalFileName, identity } from './core';
import * as debug from './debug'; import * as debug from './debug';
import { import {
compareStringsCaseInsensitive, compareStringsCaseSensitive, equateStringsCaseInsensitive, compareStringsCaseInsensitive,
equateStringsCaseSensitive, getStringComparer compareStringsCaseSensitive,
equateStringsCaseInsensitive,
equateStringsCaseSensitive,
getStringComparer
} from './stringUtils'; } from './stringUtils';
import { VirtualFileSystem } from './vfs'; import { VirtualFileSystem } from './vfs';
@ -36,7 +39,10 @@ export interface FileSystemEntries {
directories: string[]; directories: string[];
} }
export function forEachAncestorDirectory(directory: string, callback: (directory: string) => string | undefined): string | undefined { export function forEachAncestorDirectory(
directory: string,
callback: (directory: string) => string | undefined
): string | undefined {
while (true) { while (true) {
const result = callback(directory); const result = callback(directory);
if (result !== undefined) { if (result !== undefined) {
@ -58,15 +64,23 @@ export function getDirectoryPath(pathString: string): string {
export function getRootLength(pathString: string): number { export function getRootLength(pathString: string): number {
if (pathString.charAt(0) === path.sep) { if (pathString.charAt(0) === path.sep) {
if (pathString.charAt(1) !== path.sep) { return 1; } if (pathString.charAt(1) !== path.sep) {
return 1;
}
const p1 = pathString.indexOf(path.sep, 2); const p1 = pathString.indexOf(path.sep, 2);
if (p1 < 0) { return 2; } if (p1 < 0) {
return 2;
}
const p2 = pathString.indexOf(path.sep, p1 + 1); const p2 = pathString.indexOf(path.sep, p1 + 1);
if (p2 < 0) { return p1 + 1; } if (p2 < 0) {
return p1 + 1;
}
return p2 + 1; return p2 + 1;
} }
if (pathString.charAt(1) === ':') { if (pathString.charAt(1) === ':') {
if (pathString.charAt(2) === path.sep) { return 3; } if (pathString.charAt(2) === path.sep) {
return 3;
}
} }
return 0; return 0;
} }
@ -84,7 +98,9 @@ export function getPathComponents(pathString: string) {
} }
export function reducePathComponents(components: readonly string[]) { export function reducePathComponents(components: readonly string[]) {
if (!some(components)) { return []; } if (!some(components)) {
return [];
}
// Reduce the path components by eliminating // Reduce the path components by eliminating
// any '.' or '..'. // any '.' or '..'.
@ -112,7 +128,9 @@ export function reducePathComponents(components: readonly string[]) {
} }
export function combinePathComponents(components: string[]): string { export function combinePathComponents(components: string[]): string {
if (components.length === 0) { return ''; } if (components.length === 0) {
return '';
}
const root = components[0] && ensureTrailingDirectorySeparator(components[0]); const root = components[0] && ensureTrailingDirectorySeparator(components[0]);
return normalizeSlashes(root + components.slice(1).join(path.sep)); return normalizeSlashes(root + components.slice(1).join(path.sep));
@ -158,7 +176,9 @@ export function getFileSize(fs: VirtualFileSystem, path: string) {
if (stat.isFile()) { if (stat.isFile()) {
return stat.size; return stat.size;
} }
} catch { /*ignore*/ } } catch {
/*ignore*/
}
return 0; return 0;
} }
@ -242,8 +262,12 @@ export function containsPath(parent: string, child: string, currentDirectory?: s
ignoreCase = currentDirectory; ignoreCase = currentDirectory;
} }
if (parent === undefined || child === undefined) { return false; } if (parent === undefined || child === undefined) {
if (parent === child) { return true; } return false;
}
if (parent === child) {
return true;
}
const parentComponents = getPathComponents(parent); const parentComponents = getPathComponents(parent);
const childComponents = getPathComponents(child); const childComponents = getPathComponents(child);
@ -281,10 +305,22 @@ export function changeAnyExtension(path: string, ext: string): string;
* changeAnyExtension("/path/to/file.ext", ".js", [".ext", ".ts"]) === "/path/to/file.js" * changeAnyExtension("/path/to/file.ext", ".js", [".ext", ".ts"]) === "/path/to/file.js"
* ``` * ```
*/ */
export function changeAnyExtension(path: string, ext: string, extensions: string | readonly string[], ignoreCase: boolean): string; export function changeAnyExtension(
export function changeAnyExtension(path: string, ext: string, extensions?: string | readonly string[], ignoreCase?: boolean): string { path: string,
const pathext = extensions !== undefined && ignoreCase !== undefined ? ext: string,
getAnyExtensionFromPath(path, extensions, ignoreCase) : getAnyExtensionFromPath(path); extensions: string | readonly string[],
ignoreCase: boolean
): string;
export function changeAnyExtension(
path: string,
ext: string,
extensions?: string | readonly string[],
ignoreCase?: boolean
): string {
const pathext =
extensions !== undefined && ignoreCase !== undefined
? getAnyExtensionFromPath(path, extensions, ignoreCase)
: getAnyExtensionFromPath(path);
return pathext ? path.slice(0, path.length - pathext.length) + (ext.startsWith('.') ? ext : '.' + ext) : path; return pathext ? path.slice(0, path.length - pathext.length) + (ext.startsWith('.') ? ext : '.' + ext) : path;
} }
@ -309,13 +345,24 @@ export function getAnyExtensionFromPath(path: string): string;
* getAnyExtensionFromPath("/path/to/file.js", [".ext", ".js"], true) === ".js" * getAnyExtensionFromPath("/path/to/file.js", [".ext", ".js"], true) === ".js"
* getAnyExtensionFromPath("/path/to/file.ext", ".EXT", false) === "" * getAnyExtensionFromPath("/path/to/file.ext", ".EXT", false) === ""
*/ */
export function getAnyExtensionFromPath(path: string, extensions: string | readonly string[], ignoreCase: boolean): string; export function getAnyExtensionFromPath(
export function getAnyExtensionFromPath(path: string, extensions?: string | readonly string[], ignoreCase?: boolean): string { path: string,
extensions: string | readonly string[],
ignoreCase: boolean
): string;
export function getAnyExtensionFromPath(
path: string,
extensions?: string | readonly string[],
ignoreCase?: boolean
): string {
// Retrieves any string from the final "." onwards from a base file name. // Retrieves any string from the final "." onwards from a base file name.
// Unlike extensionFromPath, which throws an exception on unrecognized extensions. // Unlike extensionFromPath, which throws an exception on unrecognized extensions.
if (extensions) { if (extensions) {
return getAnyExtensionFromPathWorker(stripTrailingDirectorySeparator(path), extensions, return getAnyExtensionFromPathWorker(
ignoreCase ? equateStringsCaseInsensitive : equateStringsCaseSensitive); stripTrailingDirectorySeparator(path),
extensions,
ignoreCase ? equateStringsCaseInsensitive : equateStringsCaseSensitive
);
} }
const baseFileName = getBaseFileName(path); const baseFileName = getBaseFileName(path);
const extensionIndex = baseFileName.lastIndexOf('.'); const extensionIndex = baseFileName.lastIndexOf('.');
@ -354,20 +401,28 @@ export function getBaseFileName(pathString: string): string;
* getBaseFileName("/path/to/file.ext", ".EXT", false) === "file.ext" * getBaseFileName("/path/to/file.ext", ".EXT", false) === "file.ext"
* ``` * ```
*/ */
export function getBaseFileName(pathString: string, extensions: string | readonly string[], ignoreCase: boolean): string; export function getBaseFileName(
pathString: string,
extensions: string | readonly string[],
ignoreCase: boolean
): string;
export function getBaseFileName(pathString: string, extensions?: string | readonly string[], ignoreCase?: boolean) { export function getBaseFileName(pathString: string, extensions?: string | readonly string[], ignoreCase?: boolean) {
pathString = normalizeSlashes(pathString); pathString = normalizeSlashes(pathString);
// if the path provided is itself the root, then it has not file name. // if the path provided is itself the root, then it has not file name.
const rootLength = getRootLength(pathString); const rootLength = getRootLength(pathString);
if (rootLength === pathString.length) { return ''; } if (rootLength === pathString.length) {
return '';
}
// return the trailing portion of the path starting after the last (non-terminal) directory // return the trailing portion of the path starting after the last (non-terminal) directory
// separator but not including any trailing directory separator. // separator but not including any trailing directory separator.
pathString = stripTrailingDirectorySeparator(pathString); pathString = stripTrailingDirectorySeparator(pathString);
const name = pathString.slice(Math.max(getRootLength(pathString), pathString.lastIndexOf(path.sep) + 1)); const name = pathString.slice(Math.max(getRootLength(pathString), pathString.lastIndexOf(path.sep) + 1));
const extension = extensions !== undefined && ignoreCase !== undefined ? const extension =
getAnyExtensionFromPath(name, extensions, ignoreCase) : undefined; extensions !== undefined && ignoreCase !== undefined
? getAnyExtensionFromPath(name, extensions, ignoreCase)
: undefined;
return extension ? name.slice(0, name.length - extension.length) : name; return extension ? name.slice(0, name.length - extension.length) : name;
} }
@ -379,15 +434,29 @@ export function getRelativePathFromDirectory(from: string, to: string, ignoreCas
/** /**
* Gets a relative path that can be used to traverse between `from` and `to`. * Gets a relative path that can be used to traverse between `from` and `to`.
*/ */
export function getRelativePathFromDirectory(fromDirectory: string, to: string, getCanonicalFileName: GetCanonicalFileName): string; export function getRelativePathFromDirectory(
export function getRelativePathFromDirectory(fromDirectory: string, to: string, fromDirectory: string,
getCanonicalFileNameOrIgnoreCase: GetCanonicalFileName | boolean) { to: string,
getCanonicalFileName: GetCanonicalFileName
debug.assert((getRootLength(fromDirectory) > 0) === (getRootLength(to) > 0), 'Paths must either both be absolute or both be relative'); ): string;
const getCanonicalFileName = typeof getCanonicalFileNameOrIgnoreCase === 'function' ? getCanonicalFileNameOrIgnoreCase : identity; export function getRelativePathFromDirectory(
fromDirectory: string,
to: string,
getCanonicalFileNameOrIgnoreCase: GetCanonicalFileName | boolean
) {
debug.assert(
getRootLength(fromDirectory) > 0 === getRootLength(to) > 0,
'Paths must either both be absolute or both be relative'
);
const getCanonicalFileName =
typeof getCanonicalFileNameOrIgnoreCase === 'function' ? getCanonicalFileNameOrIgnoreCase : identity;
const ignoreCase = typeof getCanonicalFileNameOrIgnoreCase === 'boolean' ? getCanonicalFileNameOrIgnoreCase : false; const ignoreCase = typeof getCanonicalFileNameOrIgnoreCase === 'boolean' ? getCanonicalFileNameOrIgnoreCase : false;
const pathComponents = getPathComponentsRelativeTo(fromDirectory, to, ignoreCase ? const pathComponents = getPathComponentsRelativeTo(
equateStringsCaseInsensitive : equateStringsCaseSensitive, getCanonicalFileName); fromDirectory,
to,
ignoreCase ? equateStringsCaseInsensitive : equateStringsCaseSensitive,
getCanonicalFileName
);
return combinePathComponents(pathComponents); return combinePathComponents(pathComponents);
} }
@ -513,8 +582,8 @@ export function getWildcardRegexPattern(rootPath: string, fileSpec: string): str
const pathComponents = getPathComponents(absolutePath); const pathComponents = getPathComponents(absolutePath);
const escapedSeparator = getRegexEscapedSeparator(); const escapedSeparator = getRegexEscapedSeparator();
const doubleAsteriskRegexFragment = `(${ escapedSeparator }[^${ escapedSeparator }.][^${ escapedSeparator }]*)*?`; const doubleAsteriskRegexFragment = `(${escapedSeparator}[^${escapedSeparator}.][^${escapedSeparator}]*)*?`;
const reservedCharacterPattern = new RegExp(`[^\\w\\s${ escapedSeparator }]`, 'g'); const reservedCharacterPattern = new RegExp(`[^\\w\\s${escapedSeparator}]`, 'g');
// Strip the directory separator from the root component. // Strip the directory separator from the root component.
if (pathComponents.length > 0) { if (pathComponents.length > 0) {
@ -532,17 +601,16 @@ export function getWildcardRegexPattern(rootPath: string, fileSpec: string): str
component = escapedSeparator + component; component = escapedSeparator + component;
} }
regExPattern += component.replace( regExPattern += component.replace(reservedCharacterPattern, match => {
reservedCharacterPattern, match => { if (match === '*') {
if (match === '*') { return `[^${escapedSeparator}]*`;
return `[^${ escapedSeparator }]*`; } else if (match === '?') {
} else if (match === '?') { return `[^${escapedSeparator}]`;
return `[^${ escapedSeparator }]`; } else {
} else { // escaping anything that is not reserved characters - word/space/separator
// escaping anything that is not reserved characters - word/space/separator return '\\' + match;
return '\\' + match; }
} });
});
firstComponent = false; firstComponent = false;
} }
@ -591,7 +659,7 @@ export function getWildcardRoot(rootPath: string, fileSpec: string): string {
export function getFileSpec(rootPath: string, fileSpec: string): FileSpec { export function getFileSpec(rootPath: string, fileSpec: string): FileSpec {
let regExPattern = getWildcardRegexPattern(rootPath, fileSpec); let regExPattern = getWildcardRegexPattern(rootPath, fileSpec);
const escapedSeparator = getRegexEscapedSeparator(); const escapedSeparator = getRegexEscapedSeparator();
regExPattern = `^(${ regExPattern })($|${ escapedSeparator })`; regExPattern = `^(${regExPattern})($|${escapedSeparator})`;
const regExp = new RegExp(regExPattern); const regExp = new RegExp(regExPattern);
const wildcardRoot = getWildcardRoot(rootPath, fileSpec); const wildcardRoot = getWildcardRoot(rootPath, fileSpec);
@ -625,9 +693,15 @@ export function isDiskPathRoot(path: string) {
//// Path Comparisons //// Path Comparisons
function comparePathsWorker(a: string, b: string, componentComparer: (a: string, b: string) => Comparison) { function comparePathsWorker(a: string, b: string, componentComparer: (a: string, b: string) => Comparison) {
if (a === b) { return Comparison.EqualTo; } if (a === b) {
if (a === undefined) { return Comparison.LessThan; } return Comparison.EqualTo;
if (b === undefined) { return Comparison.GreaterThan; } }
if (a === undefined) {
return Comparison.LessThan;
}
if (b === undefined) {
return Comparison.GreaterThan;
}
// NOTE: Performance optimization - shortcut if the root segments differ as there would be no // NOTE: Performance optimization - shortcut if the root segments differ as there would be no
// need to perform path reduction. // need to perform path reduction.
@ -640,7 +714,7 @@ function comparePathsWorker(a: string, b: string, componentComparer: (a: string,
// check path for these segments: '', '.'. '..' // check path for these segments: '', '.'. '..'
const escapedSeparator = getRegexEscapedSeparator(); const escapedSeparator = getRegexEscapedSeparator();
const relativePathSegmentRegExp = new RegExp(`(^|${ escapedSeparator }).{0,2}($|${ escapedSeparator })`); const relativePathSegmentRegExp = new RegExp(`(^|${escapedSeparator}).{0,2}($|${escapedSeparator})`);
// NOTE: Performance optimization - shortcut if there are no relative path segments in // NOTE: Performance optimization - shortcut if there are no relative path segments in
// the non-root portion of the path // the non-root portion of the path
@ -665,21 +739,31 @@ function comparePathsWorker(a: string, b: string, componentComparer: (a: string,
return compareValues(aComponents.length, bComponents.length); return compareValues(aComponents.length, bComponents.length);
} }
function getAnyExtensionFromPathWorker(path: string, extensions: string | readonly string[], function getAnyExtensionFromPathWorker(
stringEqualityComparer: (a: string, b: string) => boolean) { path: string,
extensions: string | readonly string[],
stringEqualityComparer: (a: string, b: string) => boolean
) {
if (typeof extensions === 'string') { if (typeof extensions === 'string') {
return tryGetExtensionFromPath(path, extensions, stringEqualityComparer) || ''; return tryGetExtensionFromPath(path, extensions, stringEqualityComparer) || '';
} }
for (const extension of extensions) { for (const extension of extensions) {
const result = tryGetExtensionFromPath(path, extension, stringEqualityComparer); const result = tryGetExtensionFromPath(path, extension, stringEqualityComparer);
if (result) { return result; } if (result) {
return result;
}
} }
return ''; return '';
} }
function tryGetExtensionFromPath(path: string, extension: string, stringEqualityComparer: (a: string, b: string) => boolean) { function tryGetExtensionFromPath(
if (!extension.startsWith('.')) { extension = '.' + extension; } path: string,
extension: string,
stringEqualityComparer: (a: string, b: string) => boolean
) {
if (!extension.startsWith('.')) {
extension = '.' + extension;
}
if (path.length >= extension.length && path.charCodeAt(path.length - extension.length) === Char.Period) { if (path.length >= extension.length && path.charCodeAt(path.length - extension.length) === Char.Period) {
const pathExtension = path.slice(path.length - extension.length); const pathExtension = path.slice(path.length - extension.length);
if (stringEqualityComparer(pathExtension, extension)) { if (stringEqualityComparer(pathExtension, extension)) {
@ -690,9 +774,12 @@ function tryGetExtensionFromPath(path: string, extension: string, stringEquality
return undefined; return undefined;
} }
function getPathComponentsRelativeTo(from: string, to: string, stringEqualityComparer: (a: string, b: string) => boolean, function getPathComponentsRelativeTo(
getCanonicalFileName: GetCanonicalFileName) { from: string,
to: string,
stringEqualityComparer: (a: string, b: string) => boolean,
getCanonicalFileName: GetCanonicalFileName
) {
const fromComponents = getPathComponents(from); const fromComponents = getPathComponents(from);
const toComponents = getPathComponents(to); const toComponents = getPathComponents(to);
@ -701,7 +788,9 @@ function getPathComponentsRelativeTo(from: string, to: string, stringEqualityCom
const fromComponent = getCanonicalFileName(fromComponents[start]); const fromComponent = getCanonicalFileName(fromComponents[start]);
const toComponent = getCanonicalFileName(toComponents[start]); const toComponent = getCanonicalFileName(toComponents[start]);
const comparer = start === 0 ? equateStringsCaseInsensitive : stringEqualityComparer; const comparer = start === 0 ? equateStringsCaseInsensitive : stringEqualityComparer;
if (!comparer(fromComponent, toComponent)) { break; } if (!comparer(fromComponent, toComponent)) {
break;
}
} }
if (start === 0) { if (start === 0) {
@ -725,9 +814,12 @@ function fileSystemEntryExists(fs: VirtualFileSystem, path: string, entryKind: F
try { try {
const stat = fs.statSync(path); const stat = fs.statSync(path);
switch (entryKind) { switch (entryKind) {
case FileSystemEntryKind.File: return stat.isFile(); case FileSystemEntryKind.File:
case FileSystemEntryKind.Directory: return stat.isDirectory(); return stat.isFile();
default: return false; case FileSystemEntryKind.Directory:
return stat.isDirectory();
default:
return false;
} }
} catch (e) { } catch (e) {
return false; return false;

26
server/src/common/positionUtils.ts
View File

@ -1,12 +1,12 @@
/* /*
* positionUtils.ts * positionUtils.ts
* Copyright (c) Microsoft Corporation. * Copyright (c) Microsoft Corporation.
* Licensed under the MIT license. * Licensed under the MIT license.
* Author: Eric Traut * Author: Eric Traut
* *
* Utility routines for converting between file offsets and * Utility routines for converting between file offsets and
* line/column positions. * line/column positions.
*/ */
import { assert } from './debug'; import { assert } from './debug';
import { Position, Range, TextRange } from './textRange'; import { Position, Range, TextRange } from './textRange';
@ -38,16 +38,18 @@ export function convertOffsetToPosition(offset: number, lines: TextRangeCollecti
} }
// Translates a start/end file offset into a pair of line/column positions. // Translates a start/end file offset into a pair of line/column positions.
export function convertOffsetsToRange(startOffset: number, endOffset: number, export function convertOffsetsToRange(
lines: TextRangeCollection<TextRange>): Range { startOffset: number,
endOffset: number,
lines: TextRangeCollection<TextRange>
): Range {
const start = convertOffsetToPosition(startOffset, lines); const start = convertOffsetToPosition(startOffset, lines);
const end = convertOffsetToPosition(endOffset, lines); const end = convertOffsetToPosition(endOffset, lines);
return { start, end }; return { start, end };
} }
// Translates a position (line and col) into a file offset. // Translates a position (line and col) into a file offset.
export function convertPositionToOffset(position: Position, export function convertPositionToOffset(position: Position, lines: TextRangeCollection<TextRange>): number | undefined {
lines: TextRangeCollection<TextRange>): number | undefined {
if (position.line >= lines.count) { if (position.line >= lines.count) {
return undefined; return undefined;
} }

24
server/src/common/pythonVersion.ts
View File

@ -1,12 +1,12 @@
/* /*
* pythonLanguageVersion.ts * pythonLanguageVersion.ts
* Copyright (c) Microsoft Corporation. * Copyright (c) Microsoft Corporation.
* Licensed under the MIT license. * Licensed under the MIT license.
* Author: Eric Traut * Author: Eric Traut
* *
* Types and functions that relate to the Python language version * Types and functions that relate to the Python language version
* and features within them. * and features within them.
*/ */
export enum PythonVersion { export enum PythonVersion {
// The order of this enumeration is significant. We assume // The order of this enumeration is significant. We assume
@ -31,9 +31,9 @@ export const latestStablePythonVersion = PythonVersion.V38;
export const latestPythonVersion = PythonVersion.V38; export const latestPythonVersion = PythonVersion.V38;
export function versionToString(version: PythonVersion): string { export function versionToString(version: PythonVersion): string {
const majorVersion = (version >> 8) & 0xFF; const majorVersion = (version >> 8) & 0xff;
const minorVersion = version & 0xFF; const minorVersion = version & 0xff;
return `${ majorVersion }.${ minorVersion }`; return `${majorVersion}.${minorVersion}`;
} }
export function versionFromString(verString: string): PythonVersion | undefined { export function versionFromString(verString: string): PythonVersion | undefined {
@ -67,5 +67,5 @@ export function versionFromString(verString: string): PythonVersion | undefined
} }
export function is3x(version: PythonVersion): boolean { export function is3x(version: PythonVersion): boolean {
return (version >> 8) === 3; return version >> 8 === 3;
} }

17
server/src/common/textRange.ts
View File

@ -1,11 +1,11 @@
/* /*
* textRange.ts * textRange.ts
* Copyright (c) Microsoft Corporation. * Copyright (c) Microsoft Corporation.
* Licensed under the MIT license. * Licensed under the MIT license.
* Author: Eric Traut * Author: Eric Traut
* *
* Specifies the range of text within a larger string. * Specifies the range of text within a larger string.
*/ */
export interface TextRange { export interface TextRange {
start: number; start: number;
@ -108,8 +108,7 @@ export function doRangesOverlap(a: Range, b: Range) {
} }
export function doesRangeContain(range: Range, position: Position) { export function doesRangeContain(range: Range, position: Position) {
return comparePositions(range.start, position) >= 0 && return comparePositions(range.start, position) >= 0 && comparePositions(range.end, position) <= 0;
comparePositions(range.end, position) <= 0;
} }
export function rangesAreEqual(a: Range, b: Range) { export function rangesAreEqual(a: Range, b: Range) {

26
server/src/common/textRangeCollection.ts
View File

@ -1,15 +1,15 @@
/* /*
* textRangeCollection.ts * textRangeCollection.ts
* Copyright (c) Microsoft Corporation. * Copyright (c) Microsoft Corporation.
* Licensed under the MIT license. * Licensed under the MIT license.
* Author: Eric Traut * Author: Eric Traut
* *
* Based on code from vscode-python repository: * Based on code from vscode-python repository:
* https://github.com/Microsoft/vscode-python * https://github.com/Microsoft/vscode-python
* *
* Class that maintains an ordered list of text ranges and allows * Class that maintains an ordered list of text ranges and allows
* for indexing and fast lookups within this list. * for indexing and fast lookups within this list.
*/ */
import { TextRange } from './textRange'; import { TextRange } from './textRange';
@ -107,9 +107,7 @@ export class TextRangeCollection<T extends TextRange> {
return mid; return mid;
} }
if (mid < this.count - 1 && TextRange.getEnd(item) <= position && if (mid < this.count - 1 && TextRange.getEnd(item) <= position && position < this._items[mid + 1].start) {
position < this._items[mid + 1].start) {
return -1; return -1;
} }

18
server/src/common/timing.ts
View File

@ -1,12 +1,12 @@
/* /*
* timing.ts * timing.ts
* Copyright (c) Microsoft Corporation. * Copyright (c) Microsoft Corporation.
* Licensed under the MIT license. * Licensed under the MIT license.
* Author: Eric Traut * Author: Eric Traut
* *
* A simple duration class that can be used to record and report * A simple duration class that can be used to record and report
* durations at the millisecond level of resolution. * durations at the millisecond level of resolution.
*/ */
import { ConsoleInterface } from './console'; import { ConsoleInterface } from './console';
@ -75,7 +75,7 @@ export class TimingStats {
typeCheckerTime = new TimingStat(); typeCheckerTime = new TimingStat();
printSummary(console: ConsoleInterface) { printSummary(console: ConsoleInterface) {
console.log(`Completed in ${ this.totalDuration.getDurationInSeconds() }sec`); console.log(`Completed in ${this.totalDuration.getDurationInSeconds()}sec`);
} }
printDetails(console: ConsoleInterface) { printDetails(console: ConsoleInterface) {

45
server/src/common/vfs.ts
View File

@ -15,7 +15,11 @@ import * as fs from 'fs';
import { ConsoleInterface, NullConsole } from './console'; import { ConsoleInterface, NullConsole } from './console';
export type Listener = (eventName: 'add' | 'addDir' | 'change' | 'unlink' | 'unlinkDir', path: string, stats?: Stats) => void; export type Listener = (
eventName: 'add' | 'addDir' | 'change' | 'unlink' | 'unlinkDir',
path: string,
stats?: Stats
) => void;
export interface FileWatcher { export interface FileWatcher {
close(): void; close(): void;
@ -61,21 +65,38 @@ const _isMacintosh = process.platform === 'darwin';
const _isLinux = process.platform === 'linux'; const _isLinux = process.platform === 'linux';
class FileSystem implements VirtualFileSystem { class FileSystem implements VirtualFileSystem {
constructor(private _console: ConsoleInterface) { constructor(private _console: ConsoleInterface) {}
}
existsSync(path: string) { return fs.existsSync(path); } existsSync(path: string) {
mkdirSync(path: string) { fs.mkdirSync(path); } return fs.existsSync(path);
chdir(path: string) { process.chdir(path); } }
readdirSync(path: string) { return fs.readdirSync(path); } mkdirSync(path: string) {
fs.mkdirSync(path);
}
chdir(path: string) {
process.chdir(path);
}
readdirSync(path: string) {
return fs.readdirSync(path);
}
readFileSync(path: string, encoding?: null): Buffer; readFileSync(path: string, encoding?: null): Buffer;
readFileSync(path: string, encoding: string): string; readFileSync(path: string, encoding: string): string;
readFileSync(path: string, encoding?: string | null): Buffer | string; readFileSync(path: string, encoding?: string | null): Buffer | string;
readFileSync(path: string, encoding: string | null = null) { return fs.readFileSync(path, { encoding }); } readFileSync(path: string, encoding: string | null = null) {
writeFileSync(path: string, data: string | Buffer, encoding: string | null) { fs.writeFileSync(path, data, { encoding }); } return fs.readFileSync(path, { encoding });
statSync(path: string) { return fs.statSync(path); } }
unlinkSync(path: string) { fs.unlinkSync(path); } writeFileSync(path: string, data: string | Buffer, encoding: string | null) {
realpathSync(path: string) { return fs.realpathSync(path); } fs.writeFileSync(path, data, { encoding });
}
statSync(path: string) {
return fs.statSync(path);
}
unlinkSync(path: string) {
fs.unlinkSync(path);
}
realpathSync(path: string) {
return fs.realpathSync(path);
}
getModulePath(): string { getModulePath(): string {
// The entry point to the tool should have set the __rootDirectory // The entry point to the tool should have set the __rootDirectory

18
server/src/index.d.ts vendored
View File

@ -1,13 +1,13 @@
/* /*
* index.d.ts * index.d.ts
* Copyright (c) Microsoft Corporation. * Copyright (c) Microsoft Corporation.
* Licensed under the MIT license. * Licensed under the MIT license.
* Author: Eric Traut * Author: Eric Traut
* *
* Global definitions of extension interfaces. * Global definitions of extension interfaces.
*/ */
declare interface Promise<T> { declare interface Promise<T> {
// Catches task error and ignores them. // Catches task error and ignores them.
ignoreErrors(): void; ignoreErrors(): void;
} }

234
server/src/languageServerBase.ts
View File

@ -1,17 +1,36 @@
/* /*
* languageServerBase.ts * languageServerBase.ts
* *
* Implements common language server functionality. * Implements common language server functionality.
*/ */
import './common/extensions'; import './common/extensions';
import { import {
CodeAction, CodeActionKind, CodeActionParams, Command, CodeAction,
ConfigurationItem, createConnection, Diagnostic, DiagnosticRelatedInformation, CodeActionKind,
DiagnosticSeverity, DiagnosticTag, DocumentSymbol, ExecuteCommandParams, IConnection, CodeActionParams,
InitializeResult, Location, MarkupKind, ParameterInformation, RemoteConsole, RemoteWindow, SignatureInformation, Command,
SymbolInformation, TextDocuments, TextEdit, WorkspaceEdit ConfigurationItem,
createConnection,
Diagnostic,
DiagnosticRelatedInformation,
DiagnosticSeverity,
DiagnosticTag,
DocumentSymbol,
ExecuteCommandParams,
IConnection,
InitializeResult,
Location,
MarkupKind,
ParameterInformation,
RemoteConsole,
RemoteWindow,
SignatureInformation,
SymbolInformation,
TextDocuments,
TextEdit,
WorkspaceEdit
} from 'vscode-languageserver'; } from 'vscode-languageserver';
import { ImportResolver } from './analyzer/importResolver'; import { ImportResolver } from './analyzer/importResolver';
@ -79,12 +98,12 @@ export abstract class LanguageServerBase implements LanguageServerInterface {
fs: VirtualFileSystem; fs: VirtualFileSystem;
constructor(private _productName: string, rootDirectory: string) { constructor(private _productName: string, rootDirectory: string) {
this._connection.console.log(`${ _productName } language server starting`); this._connection.console.log(`${_productName} language server starting`);
// virtual file system to be used. initialized to real file system by default. but can't be overritten // virtual file system to be used. initialized to real file system by default. but can't be overritten
this.fs = createFromRealFileSystem(this._connection.console); this.fs = createFromRealFileSystem(this._connection.console);
// Stash the base directory into a global variable. // Stash the base directory into a global variable.
(global as any).__rootDirectory = rootDirectory; (global as any).__rootDirectory = rootDirectory;
this._connection.console.log(`Server root directory: ${ rootDirectory }`); this._connection.console.log(`Server root directory: ${rootDirectory}`);
// Create workspace map. // Create workspace map.
this._workspaceMap = new WorkspaceMap(this); this._workspaceMap = new WorkspaceMap(this);
@ -98,7 +117,9 @@ export abstract class LanguageServerBase implements LanguageServerInterface {
} }
protected abstract async executeCommand(cmdParams: ExecuteCommandParams): Promise<any>; protected abstract async executeCommand(cmdParams: ExecuteCommandParams): Promise<any>;
protected abstract async executeCodeAction(cmdParams: CodeActionParams): Promise<(Command | CodeAction)[] | undefined | null>; protected abstract async executeCodeAction(
cmdParams: CodeActionParams
): Promise<(Command | CodeAction)[] | undefined | null>;
abstract async getSettings(workspace: WorkspaceServiceInstance): Promise<ServerSettings>; abstract async getSettings(workspace: WorkspaceServiceInstance): Promise<ServerSettings>;
protected getConfiguration(workspace: WorkspaceServiceInstance, section: string) { protected getConfiguration(workspace: WorkspaceServiceInstance, section: string) {
@ -127,7 +148,7 @@ export abstract class LanguageServerBase implements LanguageServerInterface {
// Creates a service instance that's used for analyzing a // Creates a service instance that's used for analyzing a
// program within a workspace. // program within a workspace.
createAnalyzerService(name: string): AnalyzerService { createAnalyzerService(name: string): AnalyzerService {
this._connection.console.log(`Starting service instance "${ name }"`); this._connection.console.log(`Starting service instance "${name}"`);
const service = new AnalyzerService(name, this.fs, this._connection.console, this.createImportResolver); const service = new AnalyzerService(name, this.fs, this._connection.console, this.createImportResolver);
// Don't allow the analysis engine to go too long without // Don't allow the analysis engine to go too long without
@ -156,8 +177,10 @@ export abstract class LanguageServerBase implements LanguageServerInterface {
} }
const fileOrFiles = results.filesRequiringAnalysis !== 1 ? 'files' : 'file'; const fileOrFiles = results.filesRequiringAnalysis !== 1 ? 'files' : 'file';
this._connection.sendNotification('pyright/reportProgress', this._connection.sendNotification(
`${ results.filesRequiringAnalysis } ${ fileOrFiles } to analyze`); 'pyright/reportProgress',
`${results.filesRequiringAnalysis} ${fileOrFiles} to analyze`
);
} }
} else { } else {
if (this._isDisplayingProgress) { if (this._isDisplayingProgress) {
@ -184,58 +207,57 @@ export abstract class LanguageServerBase implements LanguageServerInterface {
private _setupConnection(): void { private _setupConnection(): void {
// After the server has started the client sends an initialize request. The server receives // After the server has started the client sends an initialize request. The server receives
// in the passed params the rootPath of the workspace plus the client capabilities. // in the passed params the rootPath of the workspace plus the client capabilities.
this._connection.onInitialize((params): InitializeResult => { this._connection.onInitialize(
this.rootPath = params.rootPath || ''; (params): InitializeResult => {
this.rootPath = params.rootPath || '';
// Create a service instance for each of the workspace folders. // Create a service instance for each of the workspace folders.
if (params.workspaceFolders) { if (params.workspaceFolders) {
params.workspaceFolders.forEach(folder => { params.workspaceFolders.forEach(folder => {
const path = convertUriToPath(folder.uri); const path = convertUriToPath(folder.uri);
this._workspaceMap.set(path, { this._workspaceMap.set(path, {
workspaceName: folder.name, workspaceName: folder.name,
rootPath: path, rootPath: path,
rootUri: folder.uri, rootUri: folder.uri,
serviceInstance: this.createAnalyzerService(folder.name), serviceInstance: this.createAnalyzerService(folder.name),
disableLanguageServices: false
});
});
} else if (params.rootPath) {
this._workspaceMap.set(params.rootPath, {
workspaceName: '',
rootPath: params.rootPath,
rootUri: '',
serviceInstance: this.createAnalyzerService(params.rootPath),
disableLanguageServices: false disableLanguageServices: false
}); });
});
} else if (params.rootPath) {
this._workspaceMap.set(params.rootPath, {
workspaceName: '',
rootPath: params.rootPath,
rootUri: '',
serviceInstance: this.createAnalyzerService(params.rootPath),
disableLanguageServices: false
});
}
return {
capabilities: {
// Tell the client that the server works in FULL text document
// sync mode (as opposed to incremental).
textDocumentSync: this._documents.syncKind,
definitionProvider: true,
referencesProvider: true,
documentSymbolProvider: true,
workspaceSymbolProvider: true,
hoverProvider: true,
renameProvider: true,
completionProvider: {
triggerCharacters: ['.', '['],
resolveProvider: true
},
signatureHelpProvider: {
triggerCharacters: ['(', ',', ')']
},
codeActionProvider: {
codeActionKinds: [
CodeActionKind.QuickFix,
CodeActionKind.SourceOrganizeImports
]
}
} }
};
}); return {
capabilities: {
// Tell the client that the server works in FULL text document
// sync mode (as opposed to incremental).
textDocumentSync: this._documents.syncKind,
definitionProvider: true,
referencesProvider: true,
documentSymbolProvider: true,
workspaceSymbolProvider: true,
hoverProvider: true,
renameProvider: true,
completionProvider: {
triggerCharacters: ['.', '['],
resolveProvider: true
},
signatureHelpProvider: {
triggerCharacters: ['(', ',', ')']
},
codeActionProvider: {
codeActionKinds: [CodeActionKind.QuickFix, CodeActionKind.SourceOrganizeImports]
}
}
};
}
);
this._connection.onDidChangeConfiguration(_ => { this._connection.onDidChangeConfiguration(_ => {
this._connection.console.log(`Received updated settings`); this._connection.console.log(`Received updated settings`);
@ -262,8 +284,7 @@ export abstract class LanguageServerBase implements LanguageServerInterface {
if (!locations) { if (!locations) {
return undefined; return undefined;
} }
return locations.map(loc => return locations.map(loc => Location.create(convertPathToUri(loc.path), loc.range));
Location.create(convertPathToUri(loc.path), loc.range));
}); });
this._connection.onReferences(params => { this._connection.onReferences(params => {
@ -278,13 +299,15 @@ export abstract class LanguageServerBase implements LanguageServerInterface {
if (workspace.disableLanguageServices) { if (workspace.disableLanguageServices) {
return; return;
} }
const locations = workspace.serviceInstance.getReferencesForPosition(filePath, position, const locations = workspace.serviceInstance.getReferencesForPosition(
params.context.includeDeclaration); filePath,
position,
params.context.includeDeclaration
);
if (!locations) { if (!locations) {
return undefined; return undefined;
} }
return locations.map(loc => return locations.map(loc => Location.create(convertPathToUri(loc.path), loc.range));
Location.create(convertPathToUri(loc.path), loc.range));
}); });
this._connection.onDocumentSymbol(params => { this._connection.onDocumentSymbol(params => {
@ -307,8 +330,7 @@ export abstract class LanguageServerBase implements LanguageServerInterface {
this._workspaceMap.forEach(workspace => { this._workspaceMap.forEach(workspace => {
if (!workspace.disableLanguageServices) { if (!workspace.disableLanguageServices) {
workspace.serviceInstance.addSymbolsForWorkspace( workspace.serviceInstance.addSymbolsForWorkspace(symbolList, params.query);
symbolList, params.query);
} }
}); });
@ -329,12 +351,14 @@ export abstract class LanguageServerBase implements LanguageServerInterface {
return undefined; return undefined;
} }
const markupString = hoverResults.parts.map(part => { const markupString = hoverResults.parts
if (part.python) { .map(part => {
return '```python\n' + part.text + '\n```\n'; if (part.python) {
} return '```python\n' + part.text + '\n```\n';
return part.text; }
}).join(''); return part.text;
})
.join('');
return { return {
contents: { contents: {
@ -357,8 +381,7 @@ export abstract class LanguageServerBase implements LanguageServerInterface {
if (workspace.disableLanguageServices) { if (workspace.disableLanguageServices) {
return; return;
} }
const signatureHelpResults = workspace.serviceInstance.getSignatureHelpForPosition( const signatureHelpResults = workspace.serviceInstance.getSignatureHelpForPosition(filePath, position);
filePath, position);
if (!signatureHelpResults) { if (!signatureHelpResults) {
return undefined; return undefined;
} }
@ -369,16 +392,17 @@ export abstract class LanguageServerBase implements LanguageServerInterface {
if (sig.parameters) { if (sig.parameters) {
paramInfo = sig.parameters.map(param => { paramInfo = sig.parameters.map(param => {
return ParameterInformation.create( return ParameterInformation.create(
[param.startOffset, param.endOffset], param.documentation); [param.startOffset, param.endOffset],
param.documentation
);
}); });
} }
return SignatureInformation.create(sig.label, sig.documentation, return SignatureInformation.create(sig.label, sig.documentation, ...paramInfo);
...paramInfo);
}), }),
activeSignature: signatureHelpResults.activeSignature !== undefined ? activeSignature:
signatureHelpResults.activeSignature : null, signatureHelpResults.activeSignature !== undefined ? signatureHelpResults.activeSignature : null,
activeParameter: signatureHelpResults.activeParameter !== undefined ? activeParameter:
signatureHelpResults.activeParameter : null signatureHelpResults.activeParameter !== undefined ? signatureHelpResults.activeParameter : null
}; };
}); });
@ -396,7 +420,10 @@ export abstract class LanguageServerBase implements LanguageServerInterface {
} }
const completions = workspace.serviceInstance.getCompletionsForPosition( const completions = workspace.serviceInstance.getCompletionsForPosition(
filePath, position, workspace.rootPath); filePath,
position,
workspace.rootPath
);
// Always mark as incomplete so we get called back when the // Always mark as incomplete so we get called back when the
// user continues typing. Without this, the editor will assume // user continues typing. Without this, the editor will assume
@ -414,8 +441,7 @@ export abstract class LanguageServerBase implements LanguageServerInterface {
if (completionItemData) { if (completionItemData) {
const workspace = this._workspaceMap.get(completionItemData.workspacePath); const workspace = this._workspaceMap.get(completionItemData.workspacePath);
if (workspace && completionItemData.filePath) { if (workspace && completionItemData.filePath) {
workspace.serviceInstance.resolveCompletionItem( workspace.serviceInstance.resolveCompletionItem(completionItemData.filePath, params);
completionItemData.filePath, params);
} }
} }
return params; return params;
@ -433,8 +459,7 @@ export abstract class LanguageServerBase implements LanguageServerInterface {
if (workspace.disableLanguageServices) { if (workspace.disableLanguageServices) {
return; return;
} }
const editActions = workspace.serviceInstance.renameSymbolAtPosition( const editActions = workspace.serviceInstance.renameSymbolAtPosition(filePath, position, params.newName);
filePath, position, params.newName);
if (!editActions) { if (!editActions) {
return undefined; return undefined;
@ -462,10 +487,7 @@ export abstract class LanguageServerBase implements LanguageServerInterface {
this._connection.onDidOpenTextDocument(params => { this._connection.onDidOpenTextDocument(params => {
const filePath = convertUriToPath(params.textDocument.uri); const filePath = convertUriToPath(params.textDocument.uri);
const service = this._workspaceMap.getWorkspaceForFile(filePath).serviceInstance; const service = this._workspaceMap.getWorkspaceForFile(filePath).serviceInstance;
service.setFileOpened( service.setFileOpened(filePath, params.textDocument.version, params.textDocument.text);
filePath,
params.textDocument.version,
params.textDocument.text);
}); });
this._connection.onDidChangeTextDocument(params => { this._connection.onDidChangeTextDocument(params => {
@ -473,10 +495,7 @@ export abstract class LanguageServerBase implements LanguageServerInterface {
const filePath = convertUriToPath(params.textDocument.uri); const filePath = convertUriToPath(params.textDocument.uri);
const service = this._workspaceMap.getWorkspaceForFile(filePath).serviceInstance; const service = this._workspaceMap.getWorkspaceForFile(filePath).serviceInstance;
service.updateOpenFileContents( service.updateOpenFileContents(filePath, params.textDocument.version, params.contentChanges[0].text);
filePath,
params.textDocument.version,
params.contentChanges[0].text);
}); });
this._connection.onDidCloseTextDocument(params => { this._connection.onDidCloseTextDocument(params => {
@ -522,25 +541,26 @@ export abstract class LanguageServerBase implements LanguageServerInterface {
workspace.disableLanguageServices = !!serverSettings.disableLanguageServices; workspace.disableLanguageServices = !!serverSettings.disableLanguageServices;
} }
updateOptionsAndRestartService(workspace: WorkspaceServiceInstance, updateOptionsAndRestartService(
serverSettings: ServerSettings, typeStubTargetImportName?: string) { workspace: WorkspaceServiceInstance,
serverSettings: ServerSettings,
typeStubTargetImportName?: string
) {
AnalyzerServiceExecutor.runWithOptions(this.rootPath, workspace, serverSettings, typeStubTargetImportName); AnalyzerServiceExecutor.runWithOptions(this.rootPath, workspace, serverSettings, typeStubTargetImportName);
} }
private _convertDiagnostics(diags: AnalyzerDiagnostic[]): Diagnostic[] { private _convertDiagnostics(diags: AnalyzerDiagnostic[]): Diagnostic[] {
return diags.map(diag => { return diags.map(diag => {
const severity = diag.category === DiagnosticCategory.Error ? const severity =
DiagnosticSeverity.Error : DiagnosticSeverity.Warning; diag.category === DiagnosticCategory.Error ? DiagnosticSeverity.Error : DiagnosticSeverity.Warning;
let source = this._productName; let source = this._productName;
const rule = diag.getRule(); const rule = diag.getRule();
if (rule) { if (rule) {
source = `${ source } (${ rule })`; source = `${source} (${rule})`;
} }
const vsDiag = Diagnostic.create(diag.range, diag.message, severity, const vsDiag = Diagnostic.create(diag.range, diag.message, severity, undefined, source);
undefined, source);
if (diag.category === DiagnosticCategory.UnusedCode) { if (diag.category === DiagnosticCategory.UnusedCode) {
vsDiag.tags = [DiagnosticTag.Unnecessary]; vsDiag.tags = [DiagnosticTag.Unnecessary];

38
server/src/languageService/analyzerServiceExecutor.ts
View File

@ -12,20 +12,30 @@ import { combinePaths, normalizePath } from '../common/pathUtils';
import { ServerSettings, WorkspaceServiceInstance } from '../languageServerBase'; import { ServerSettings, WorkspaceServiceInstance } from '../languageServerBase';
export class AnalyzerServiceExecutor { export class AnalyzerServiceExecutor {
static runWithOptions(languageServiceRootPath: string, workspace: WorkspaceServiceInstance, static runWithOptions(
serverSettings: ServerSettings, typeStubTargetImportName?: string): void { languageServiceRootPath: string,
workspace: WorkspaceServiceInstance,
const commandLineOptions = GetCommandLineOptions(languageServiceRootPath, workspace.rootPath, serverSettings: ServerSettings,
serverSettings, typeStubTargetImportName); typeStubTargetImportName?: string
): void {
const commandLineOptions = GetCommandLineOptions(
languageServiceRootPath,
workspace.rootPath,
serverSettings,
typeStubTargetImportName
);
// setting option cause analyzer service to re-analyze everything // setting option cause analyzer service to re-analyze everything
workspace.serviceInstance.setOptions(commandLineOptions); workspace.serviceInstance.setOptions(commandLineOptions);
} }
} }
function GetCommandLineOptions(languageServiceRootPath: string, workspaceRootPath: string, function GetCommandLineOptions(
serverSettings: ServerSettings, typeStubTargetImportName?: string) { languageServiceRootPath: string,
workspaceRootPath: string,
serverSettings: ServerSettings,
typeStubTargetImportName?: string
) {
const commandLineOptions = new CommandLineOptions(workspaceRootPath, true); const commandLineOptions = new CommandLineOptions(workspaceRootPath, true);
commandLineOptions.checkOnlyOpenFiles = serverSettings.openFilesOnly; commandLineOptions.checkOnlyOpenFiles = serverSettings.openFilesOnly;
commandLineOptions.useLibraryCodeForTypes = serverSettings.useLibraryCodeForTypes; commandLineOptions.useLibraryCodeForTypes = serverSettings.useLibraryCodeForTypes;
@ -37,8 +47,10 @@ function GetCommandLineOptions(languageServiceRootPath: string, workspaceRootPat
commandLineOptions.watch = !commandLineOptions.checkOnlyOpenFiles; commandLineOptions.watch = !commandLineOptions.checkOnlyOpenFiles;
if (serverSettings.venvPath) { if (serverSettings.venvPath) {
commandLineOptions.venvPath = combinePaths(workspaceRootPath || languageServiceRootPath, commandLineOptions.venvPath = combinePaths(
normalizePath(_expandPathVariables(languageServiceRootPath, serverSettings.venvPath))); workspaceRootPath || languageServiceRootPath,
normalizePath(_expandPathVariables(languageServiceRootPath, serverSettings.venvPath))
);
} }
if (serverSettings.pythonPath) { if (serverSettings.pythonPath) {
@ -46,8 +58,10 @@ function GetCommandLineOptions(languageServiceRootPath: string, workspaceRootPat
// the local python interpreter should be used rather than interpreting the // the local python interpreter should be used rather than interpreting the
// setting value as a path to the interpreter. We'll simply ignore it in this case. // setting value as a path to the interpreter. We'll simply ignore it in this case.
if (serverSettings.pythonPath.trim() !== 'python') { if (serverSettings.pythonPath.trim() !== 'python') {
commandLineOptions.pythonPath = combinePaths(workspaceRootPath || languageServiceRootPath, commandLineOptions.pythonPath = combinePaths(
normalizePath(_expandPathVariables(languageServiceRootPath, serverSettings.pythonPath))); workspaceRootPath || languageServiceRootPath,
normalizePath(_expandPathVariables(languageServiceRootPath, serverSettings.pythonPath))
);
} }
} }

40
server/src/languageService/codeActionProvider.ts
View File

@ -14,8 +14,10 @@ import { WorkspaceServiceInstance } from '../languageServerBase';
export class CodeActionProvider { export class CodeActionProvider {
static getCodeActionsForPosition(workspace: WorkspaceServiceInstance, filePath: string, range: Range) { static getCodeActionsForPosition(workspace: WorkspaceServiceInstance, filePath: string, range: Range) {
const sortImportsCodeAction = CodeAction.create( const sortImportsCodeAction = CodeAction.create(
'Organize Imports', Command.create('Organize Imports', Commands.orderImports), 'Organize Imports',
CodeActionKind.SourceOrganizeImports); Command.create('Organize Imports', Commands.orderImports),
CodeActionKind.SourceOrganizeImports
);
const codeActions: CodeAction[] = [sortImportsCodeAction]; const codeActions: CodeAction[] = [sortImportsCodeAction];
if (!workspace.disableLanguageServices) { if (!workspace.disableLanguageServices) {
@ -26,14 +28,21 @@ export class CodeActionProvider {
}); });
if (typeStubDiag) { if (typeStubDiag) {
const action = typeStubDiag.getActions()!.find( const action = typeStubDiag
a => a.action === Commands.createTypeStub) as CreateTypeStubFileAction; .getActions()!
.find(a => a.action === Commands.createTypeStub) as CreateTypeStubFileAction;
if (action) { if (action) {
const createTypeStubAction = CodeAction.create( const createTypeStubAction = CodeAction.create(
`Create Type Stub For ‘${ action.moduleName }’`, `Create Type Stub For '${action.moduleName}'`,
Command.create('Create Type Stub', Commands.createTypeStub, Command.create(
workspace.rootPath, action.moduleName, filePath), 'Create Type Stub',
CodeActionKind.QuickFix); Commands.createTypeStub,
workspace.rootPath,
action.moduleName,
filePath
),
CodeActionKind.QuickFix
);
codeActions.push(createTypeStubAction); codeActions.push(createTypeStubAction);
} }
} }
@ -44,14 +53,19 @@ export class CodeActionProvider {
}); });
if (addOptionalDiag) { if (addOptionalDiag) {
const action = addOptionalDiag.getActions()!.find( const action = addOptionalDiag
a => a.action === Commands.addMissingOptionalToParam) as AddMissingOptionalToParamAction; .getActions()!
.find(a => a.action === Commands.addMissingOptionalToParam) as AddMissingOptionalToParamAction;
if (action) { if (action) {
const addMissingOptionalAction = CodeAction.create( const addMissingOptionalAction = CodeAction.create(
`Add 'Optional' to type annotation`, `Add 'Optional' to type annotation`,
Command.create(`Add 'Optional' to type annotation`, Commands.addMissingOptionalToParam, Command.create(
action.offsetOfTypeNode), `Add 'Optional' to type annotation`,
CodeActionKind.QuickFix); Commands.addMissingOptionalToParam,
action.offsetOfTypeNode
),
CodeActionKind.QuickFix
);
codeActions.push(addMissingOptionalAction); codeActions.push(addMissingOptionalAction);
} }
} }

498
server/src/languageService/completionProvider.ts
View File

@ -1,15 +1,14 @@
/* /*
* completionProvider.ts * completionProvider.ts
* Copyright (c) Microsoft Corporation. * Copyright (c) Microsoft Corporation.
* Licensed under the MIT license. * Licensed under the MIT license.
* Author: Eric Traut * Author: Eric Traut
* *
* Logic that maps a position within a Python program file into * Logic that maps a position within a Python program file into
* a list of zero or more text completions that apply in the context. * a list of zero or more text completions that apply in the context.
*/ */
import { CompletionItem, CompletionItemKind, CompletionList, import { CompletionItem, CompletionItemKind, CompletionList, MarkupKind, Range, TextEdit } from 'vscode-languageserver';
MarkupKind, Range, TextEdit } from 'vscode-languageserver';
import { ImportLookup } from '../analyzer/analyzerFileInfo'; import { ImportLookup } from '../analyzer/analyzerFileInfo';
import * as AnalyzerNodeInfo from '../analyzer/analyzerNodeInfo'; import * as AnalyzerNodeInfo from '../analyzer/analyzerNodeInfo';
@ -22,8 +21,8 @@ import * as ParseTreeUtils from '../analyzer/parseTreeUtils';
import { Symbol, SymbolTable } from '../analyzer/symbol'; import { Symbol, SymbolTable } from '../analyzer/symbol';
import * as SymbolNameUtils from '../analyzer/symbolNameUtils'; import * as SymbolNameUtils from '../analyzer/symbolNameUtils';
import { getLastTypedDeclaredForSymbol } from '../analyzer/symbolUtils'; import { getLastTypedDeclaredForSymbol } from '../analyzer/symbolUtils';
import { CallSignatureInfo,TypeEvaluator } from '../analyzer/typeEvaluator'; import { CallSignatureInfo, TypeEvaluator } from '../analyzer/typeEvaluator';
import { ClassType, FunctionType, Type,TypeCategory } from '../analyzer/types'; import { ClassType, FunctionType, Type, TypeCategory } from '../analyzer/types';
import { doForSubtypes, getMembersForClass, getMembersForModule } from '../analyzer/typeUtils'; import { doForSubtypes, getMembersForClass, getMembersForModule } from '../analyzer/typeUtils';
import { ConfigOptions } from '../common/configOptions'; import { ConfigOptions } from '../common/configOptions';
import { TextEditAction } from '../common/editAction'; import { TextEditAction } from '../common/editAction';
@ -32,9 +31,20 @@ import { convertOffsetToPosition, convertPositionToOffset } from '../common/posi
import * as StringUtils from '../common/stringUtils'; import * as StringUtils from '../common/stringUtils';
import { comparePositions, Position } from '../common/textRange'; import { comparePositions, Position } from '../common/textRange';
import { TextRange } from '../common/textRange'; import { TextRange } from '../common/textRange';
import { ErrorExpressionCategory, ErrorNode, ExpressionNode, import {
FunctionNode, ImportFromNode, isExpressionNode, ModuleNameNode, NameNode, ErrorExpressionCategory,
ParameterCategory, ParseNode, ParseNodeType, StringNode } from '../parser/parseNodes'; ErrorNode,
ExpressionNode,
FunctionNode,
ImportFromNode,
isExpressionNode,
ModuleNameNode,
NameNode,
ParameterCategory,
ParseNode,
ParseNodeType,
StringNode
} from '../parser/parseNodes';
import { ParseResults } from '../parser/parser'; import { ParseResults } from '../parser/parser';
const _keywords: string[] = [ const _keywords: string[] = [
@ -161,8 +171,8 @@ export class CompletionProvider {
private _configOptions: ConfigOptions, private _configOptions: ConfigOptions,
private _importLookup: ImportLookup, private _importLookup: ImportLookup,
private _evaluator: TypeEvaluator, private _evaluator: TypeEvaluator,
private _moduleSymbolsCallback: () => ModuleSymbolMap) { private _moduleSymbolsCallback: () => ModuleSymbolMap
} ) {}
getCompletionsForPosition(): CompletionList | undefined { getCompletionsForPosition(): CompletionList | undefined {
const offset = convertPositionToOffset(this._position, this._parseResults.tokenizerOutput.lines); const offset = convertPositionToOffset(this._position, this._parseResults.tokenizerOutput.lines);
@ -244,8 +254,7 @@ export class CompletionProvider {
} }
if (curNode.nodeType === ParseNodeType.Error) { if (curNode.nodeType === ParseNodeType.Error) {
return this._getExpressionErrorCompletions(curNode, priorWord, return this._getExpressionErrorCompletions(curNode, priorWord, priorText, postText);
priorText, postText);
} }
if (curNode.nodeType === ParseNodeType.MemberAccess) { if (curNode.nodeType === ParseNodeType.MemberAccess) {
@ -268,8 +277,7 @@ export class CompletionProvider {
} }
} }
} else if (curNode.parent && curNode.parent.nodeType === ParseNodeType.MemberAccess) { } else if (curNode.parent && curNode.parent.nodeType === ParseNodeType.MemberAccess) {
return this._getMemberAccessCompletions( return this._getMemberAccessCompletions(curNode.parent.leftExpression, priorWord);
curNode.parent.leftExpression, priorWord);
} }
} }
@ -308,8 +316,8 @@ export class CompletionProvider {
} }
const curIndex = CompletionProvider._mostRecentCompletions.findIndex( const curIndex = CompletionProvider._mostRecentCompletions.findIndex(
item => item.label === label && item => item.label === label && item.autoImportText === autoImportText
item.autoImportText === autoImportText); );
if (curIndex > 0) { if (curIndex > 0) {
// If there's an existing entry with the same name that's not at the // If there's an existing entry with the same name that's not at the
@ -356,9 +364,12 @@ export class CompletionProvider {
return !!priorText.match(/#/); return !!priorText.match(/#/);
} }
private _getExpressionErrorCompletions(node: ErrorNode, priorWord: string, private _getExpressionErrorCompletions(
priorText: string, postText: string): CompletionList | undefined { node: ErrorNode,
priorWord: string,
priorText: string,
postText: string
): CompletionList | undefined {
// Is the error due to a missing member access name? If so, // Is the error due to a missing member access name? If so,
// we can evaluate the left side of the member access expression // we can evaluate the left side of the member access expression
// to determine its type and offer suggestions based on it. // to determine its type and offer suggestions based on it.
@ -400,8 +411,7 @@ export class CompletionProvider {
private _createSingleKeywordCompletionList(keyword: string): CompletionList { private _createSingleKeywordCompletionList(keyword: string): CompletionList {
const completionItem = CompletionItem.create(keyword); const completionItem = CompletionItem.create(keyword);
completionItem.kind = CompletionItemKind.Keyword; completionItem.kind = CompletionItemKind.Keyword;
completionItem.sortText = completionItem.sortText = this._makeSortText(SortCategory.LikelyKeyword, keyword);
this._makeSortText(SortCategory.LikelyKeyword, keyword);
return CompletionList.create([completionItem]); return CompletionList.create([completionItem]);
} }
@ -439,8 +449,7 @@ export class CompletionProvider {
const methodSignature = this._printMethodSignature(decl.node) + ':'; const methodSignature = this._printMethodSignature(decl.node) + ':';
const textEdit = TextEdit.replace(range, methodSignature); const textEdit = TextEdit.replace(range, methodSignature);
this._addSymbol(name, symbol, partialName.value, completionList, this._addSymbol(name, symbol, partialName.value, completionList, undefined, textEdit);
undefined, textEdit);
} }
} }
}); });
@ -449,24 +458,26 @@ export class CompletionProvider {
} }
private _printMethodSignature(node: FunctionNode): string { private _printMethodSignature(node: FunctionNode): string {
const paramList = node.parameters.map(param => { const paramList = node.parameters
let paramString = ''; .map(param => {
if (param.category === ParameterCategory.VarArgList) { let paramString = '';
paramString += '*'; if (param.category === ParameterCategory.VarArgList) {
} else if (param.category === ParameterCategory.VarArgDictionary) { paramString += '*';
paramString += '**'; } else if (param.category === ParameterCategory.VarArgDictionary) {
} paramString += '**';
}
if (param.name) { if (param.name) {
paramString += param.name.value; paramString += param.name.value;
} }
if (param.typeAnnotation) { if (param.typeAnnotation) {
paramString += ': ' + ParseTreeUtils.printExpression(param.typeAnnotation); paramString += ': ' + ParseTreeUtils.printExpression(param.typeAnnotation);
} }
return paramString; return paramString;
}).join(', '); })
.join(', ');
let methodSignature = node.name.value + '(' + paramList + ')'; let methodSignature = node.name.value + '(' + paramList + ')';
@ -477,9 +488,7 @@ export class CompletionProvider {
return methodSignature; return methodSignature;
} }
private _getMemberAccessCompletions(leftExprNode: ExpressionNode, private _getMemberAccessCompletions(leftExprNode: ExpressionNode, priorWord: string): CompletionList | undefined {
priorWord: string): CompletionList | undefined {
const leftType = this._evaluator.getType(leftExprNode); const leftType = this._evaluator.getType(leftExprNode);
const symbolTable = new Map<string, Symbol>(); const symbolTable = new Map<string, Symbol>();
@ -498,23 +507,27 @@ export class CompletionProvider {
} }
const completionList = CompletionList.create(); const completionList = CompletionList.create();
this._addSymbolsForSymbolTable(symbolTable, _ => true, this._addSymbolsForSymbolTable(symbolTable, _ => true, priorWord, completionList);
priorWord, completionList);
return completionList; return completionList;
} }
private _getStatementCompletions(parseNode: ParseNode, priorWord: string, private _getStatementCompletions(
priorText: string, postText: string): CompletionList | undefined { parseNode: ParseNode,
priorWord: string,
priorText: string,
postText: string
): CompletionList | undefined {
// For now, use the same logic for expressions and statements. // For now, use the same logic for expressions and statements.
return this._getExpressionCompletions(parseNode, priorWord, return this._getExpressionCompletions(parseNode, priorWord, priorText, postText);
priorText, postText);
} }
private _getExpressionCompletions(parseNode: ParseNode, priorWord: string, private _getExpressionCompletions(
priorText: string, postText: string): CompletionList | undefined { parseNode: ParseNode,
priorWord: string,
priorText: string,
postText: string
): CompletionList | undefined {
// If the user typed a "." as part of a number, don't present // If the user typed a "." as part of a number, don't present
// any completion options. // any completion options.
if (parseNode.nodeType === ParseNodeType.Number) { if (parseNode.nodeType === ParseNodeType.Number) {
@ -534,8 +547,7 @@ export class CompletionProvider {
const completionItem = CompletionItem.create(keyword); const completionItem = CompletionItem.create(keyword);
completionItem.kind = CompletionItemKind.Keyword; completionItem.kind = CompletionItemKind.Keyword;
completionList.items.push(completionItem); completionList.items.push(completionItem);
completionItem.sortText = completionItem.sortText = this._makeSortText(SortCategory.Keyword, keyword);
this._makeSortText(SortCategory.Keyword, keyword);
}); });
// Add auto-import suggestions from other modules. // Add auto-import suggestions from other modules.
@ -551,11 +563,11 @@ export class CompletionProvider {
} else if (parseNode.category === ErrorExpressionCategory.MissingExpression) { } else if (parseNode.category === ErrorExpressionCategory.MissingExpression) {
if (parseNode.parent && parseNode.parent.nodeType === ParseNodeType.Assignment) { if (parseNode.parent && parseNode.parent.nodeType === ParseNodeType.Assignment) {
const declaredTypeOfTarget = this._evaluator.getDeclaredTypeForExpression( const declaredTypeOfTarget = this._evaluator.getDeclaredTypeForExpression(
parseNode.parent.leftExpression); parseNode.parent.leftExpression
);
if (declaredTypeOfTarget) { if (declaredTypeOfTarget) {
this._addLiteralValuesForTargetType(declaredTypeOfTarget, this._addLiteralValuesForTargetType(declaredTypeOfTarget, priorText, postText, completionList);
priorText, postText, completionList);
} }
} }
} }
@ -564,20 +576,23 @@ export class CompletionProvider {
return completionList; return completionList;
} }
private _addCallArgumentCompletions(parseNode: ParseNode, priorWord: string, priorText: string, private _addCallArgumentCompletions(
postText: string, completionList: CompletionList) { parseNode: ParseNode,
priorWord: string,
priorText: string,
postText: string,
completionList: CompletionList
) {
// If we're within the argument list of a call, add parameter names. // If we're within the argument list of a call, add parameter names.
const offset = convertPositionToOffset(this._position, const offset = convertPositionToOffset(this._position, this._parseResults.tokenizerOutput.lines)!;
this._parseResults.tokenizerOutput.lines)!; const signatureInfo = this._evaluator.getCallSignatureInfo(parseNode, offset);
const signatureInfo = this._evaluator.getCallSignatureInfo(parseNode, offset)
if (signatureInfo) { if (signatureInfo) {
// Are we past the call expression and within the argument list? // Are we past the call expression and within the argument list?
const callNameEnd = convertOffsetToPosition( const callNameEnd = convertOffsetToPosition(
signatureInfo.callNode.leftExpression.start + signatureInfo.callNode.leftExpression.start + signatureInfo.callNode.leftExpression.length,
signatureInfo.callNode.leftExpression.length, this._parseResults.tokenizerOutput.lines
this._parseResults.tokenizerOutput.lines); );
if (comparePositions(this._position, callNameEnd) > 0) { if (comparePositions(this._position, callNameEnd) > 0) {
this._addNamedParameters(signatureInfo, priorWord, completionList); this._addNamedParameters(signatureInfo, priorWord, completionList);
@ -588,15 +603,19 @@ export class CompletionProvider {
} }
} }
private _addLiteralValuesForArgument(signatureInfo: CallSignatureInfo, private _addLiteralValuesForArgument(
priorText: string, postText: string, completionList: CompletionList) { signatureInfo: CallSignatureInfo,
priorText: string,
postText: string,
completionList: CompletionList
) {
signatureInfo.signatures.forEach(signature => { signatureInfo.signatures.forEach(signature => {
let paramIndex = -1; let paramIndex = -1;
if (signatureInfo.activeArgumentName !== undefined) { if (signatureInfo.activeArgumentName !== undefined) {
paramIndex = signature.details.parameters.findIndex(param => { paramIndex = signature.details.parameters.findIndex(param => {
param.name === signatureInfo.activeArgumentName}); param.name === signatureInfo.activeArgumentName;
});
} else if (signatureInfo.activeArgumentIndex < signature.details.parameters.length) { } else if (signatureInfo.activeArgumentIndex < signature.details.parameters.length) {
paramIndex = signatureInfo.activeArgumentIndex; paramIndex = signatureInfo.activeArgumentIndex;
} }
@ -611,17 +630,24 @@ export class CompletionProvider {
}); });
} }
private _addLiteralValuesForTargetType(type: Type, priorText: string, postText: string, private _addLiteralValuesForTargetType(
completionList: CompletionList) { type: Type,
priorText: string,
postText: string,
completionList: CompletionList
) {
const quoteValue = this._getQuoteValueFromPriorText(priorText); const quoteValue = this._getQuoteValueFromPriorText(priorText);
doForSubtypes(type, subtype => { doForSubtypes(type, subtype => {
if (subtype.category === TypeCategory.Object) { if (subtype.category === TypeCategory.Object) {
if (ClassType.isBuiltIn(subtype.classType, 'str')) { if (ClassType.isBuiltIn(subtype.classType, 'str')) {
if (subtype.literalValue !== undefined) { if (subtype.literalValue !== undefined) {
this._addStringLiteralToCompletionList(subtype.literalValue as string, this._addStringLiteralToCompletionList(
quoteValue.stringValue, postText, quoteValue.quoteCharacter, subtype.literalValue as string,
completionList); quoteValue.stringValue,
postText,
quoteValue.quoteCharacter,
completionList
);
} }
} }
} }
@ -630,9 +656,12 @@ export class CompletionProvider {
}); });
} }
private _getStringLiteralCompletions(parseNode: StringNode, priorWord: string, private _getStringLiteralCompletions(
priorText: string, postText: string): CompletionList | undefined { parseNode: StringNode,
priorWord: string,
priorText: string,
postText: string
): CompletionList | undefined {
let parentNode: ParseNode | undefined = parseNode.parent; let parentNode: ParseNode | undefined = parseNode.parent;
if (!parentNode || parentNode.nodeType !== ParseNodeType.StringList || parentNode.strings.length > 1) { if (!parentNode || parentNode.nodeType !== ParseNodeType.StringList || parentNode.strings.length > 1) {
return undefined; return undefined;
@ -655,7 +684,7 @@ export class CompletionProvider {
if (!baseType || baseType.category !== TypeCategory.Object) { if (!baseType || baseType.category !== TypeCategory.Object) {
return undefined; return undefined;
} }
// We currently handle only TypedDict objects. // We currently handle only TypedDict objects.
const classType = baseType.classType; const classType = baseType.classType;
if (!ClassType.isTypedDictClass(classType)) { if (!ClassType.isTypedDictClass(classType)) {
@ -666,16 +695,19 @@ export class CompletionProvider {
const quoteValue = this._getQuoteValueFromPriorText(priorText); const quoteValue = this._getQuoteValueFromPriorText(priorText);
entries.forEach((_, key) => { entries.forEach((_, key) => {
this._addStringLiteralToCompletionList(key, quoteValue.stringValue, postText, this._addStringLiteralToCompletionList(
quoteValue.quoteCharacter, completionList); key,
quoteValue.stringValue,
postText,
quoteValue.quoteCharacter,
completionList
);
}); });
} else if (parentNode.nodeType === ParseNodeType.Assignment) { } else if (parentNode.nodeType === ParseNodeType.Assignment) {
const declaredTypeOfTarget = this._evaluator.getDeclaredTypeForExpression( const declaredTypeOfTarget = this._evaluator.getDeclaredTypeForExpression(parentNode.leftExpression);
parentNode.leftExpression);
if (declaredTypeOfTarget) { if (declaredTypeOfTarget) {
this._addLiteralValuesForTargetType(declaredTypeOfTarget, this._addLiteralValuesForTargetType(declaredTypeOfTarget, priorText, postText, completionList);
priorText, postText, completionList);
} }
} else { } else {
this._addCallArgumentCompletions(parseNode, priorWord, priorText, postText, completionList); this._addCallArgumentCompletions(parseNode, priorWord, priorText, postText, completionList);
@ -689,14 +721,14 @@ export class CompletionProvider {
// (either starting with a single or double quote). Returns the quote // (either starting with a single or double quote). Returns the quote
// type and the string literal value after the starting quote. // type and the string literal value after the starting quote.
private _getQuoteValueFromPriorText(priorText: string) { private _getQuoteValueFromPriorText(priorText: string) {
const lastSingleQuote = priorText.lastIndexOf('\''); const lastSingleQuote = priorText.lastIndexOf("'");
const lastDoubleQuote = priorText.lastIndexOf('"'); const lastDoubleQuote = priorText.lastIndexOf('"');
let quoteCharacter = this._parseResults.tokenizerOutput.predominantSingleQuoteCharacter; let quoteCharacter = this._parseResults.tokenizerOutput.predominantSingleQuoteCharacter;
let stringValue = undefined; let stringValue = undefined;
if (lastSingleQuote > lastDoubleQuote) { if (lastSingleQuote > lastDoubleQuote) {
quoteCharacter = '\''; quoteCharacter = "'";
stringValue = priorText.substr(lastSingleQuote + 1); stringValue = priorText.substr(lastSingleQuote + 1);
} else if (lastDoubleQuote > lastSingleQuote) { } else if (lastDoubleQuote > lastSingleQuote) {
quoteCharacter = '"'; quoteCharacter = '"';
@ -729,18 +761,26 @@ export class CompletionProvider {
const entries = this._evaluator.getTypedDictMembersForClass(classType); const entries = this._evaluator.getTypedDictMembersForClass(classType);
entries.forEach((_, key) => { entries.forEach((_, key) => {
this._addStringLiteralToCompletionList(key, undefined, undefined, this._addStringLiteralToCompletionList(
key,
undefined,
undefined,
this._parseResults.tokenizerOutput.predominantSingleQuoteCharacter, this._parseResults.tokenizerOutput.predominantSingleQuoteCharacter,
completionList); completionList
);
}); });
} }
private _addStringLiteralToCompletionList(value: string, priorString: string | undefined, private _addStringLiteralToCompletionList(
postText: string | undefined, quoteCharacter: string, completionList: CompletionList) { value: string,
priorString: string | undefined,
postText: string | undefined,
quoteCharacter: string,
completionList: CompletionList
) {
const isSimilar = StringUtils.computeCompletionSimilarity(priorString || '', value) > similarityLimit; const isSimilar = StringUtils.computeCompletionSimilarity(priorString || '', value) > similarityLimit;
if (isSimilar) { if (isSimilar) {
const valueWithQuotes = `${ quoteCharacter }${ value }${ quoteCharacter }`; const valueWithQuotes = `${quoteCharacter}${value}${quoteCharacter}`;
const completionItem = CompletionItem.create(valueWithQuotes); const completionItem = CompletionItem.create(valueWithQuotes);
completionItem.kind = CompletionItemKind.Text; completionItem.kind = CompletionItemKind.Text;
@ -767,12 +807,11 @@ export class CompletionProvider {
completionList.items.push(completionItem); completionList.items.push(completionItem);
} }
} }
private _getAutoImportCompletions(priorWord: string, completionList: CompletionList) { private _getAutoImportCompletions(priorWord: string, completionList: CompletionList) {
const moduleSymbolMap = this._moduleSymbolsCallback(); const moduleSymbolMap = this._moduleSymbolsCallback();
const importStatements = ImportStatementUtils.getTopLevelImports( const importStatements = ImportStatementUtils.getTopLevelImports(this._parseResults.parseTree);
this._parseResults.parseTree);
moduleSymbolMap.forEach((symbolTable, filePath) => { moduleSymbolMap.forEach((symbolTable, filePath) => {
const fileName = stripFileExtension(getFileName(filePath)); const fileName = stripFileExtension(getFileName(filePath));
@ -784,9 +823,10 @@ export class CompletionProvider {
// For very short matching strings, we will require an exact match. Otherwise // For very short matching strings, we will require an exact match. Otherwise
// we will tend to return a list that's too long. Once we get beyond two // we will tend to return a list that's too long. Once we get beyond two
// characters, we can do a fuzzy match. // characters, we can do a fuzzy match.
const isSimilar = priorWord.length > 2 ? const isSimilar =
StringUtils.computeCompletionSimilarity(priorWord, name) > similarityLimit : priorWord.length > 2
priorWord.length > 0 && name.startsWith(priorWord); ? StringUtils.computeCompletionSimilarity(priorWord, name) > similarityLimit
: priorWord.length > 0 && name.startsWith(priorWord);
if (isSimilar) { if (isSimilar) {
if (!symbol.isExternallyHidden()) { if (!symbol.isExternallyHidden()) {
@ -794,7 +834,8 @@ export class CompletionProvider {
// this name, don't add an auto-import suggestion with // this name, don't add an auto-import suggestion with
// the same name. // the same name.
const localDuplicate = completionList.items.find( const localDuplicate = completionList.items.find(
item => item.label === name && !item.data.autoImport); item => item.label === name && !item.data.autoImport
);
const declarations = symbol.getDeclarations(); const declarations = symbol.getDeclarations();
if (declarations && declarations.length > 0 && localDuplicate === undefined) { if (declarations && declarations.length > 0 && localDuplicate === undefined) {
// Don't include imported symbols, only those that // Don't include imported symbols, only those that
@ -812,11 +853,22 @@ export class CompletionProvider {
} }
const autoImportTextEdits = this._getTextEditsForAutoImportByFilePath( const autoImportTextEdits = this._getTextEditsForAutoImportByFilePath(
name, importStatements, filePath, importSource, name,
moduleNameAndType ? moduleNameAndType.importType : ImportType.Local); importStatements,
filePath,
importSource,
moduleNameAndType ? moduleNameAndType.importType : ImportType.Local
);
this._addSymbol(name, symbol, priorWord, this._addSymbol(
completionList, importSource, undefined, autoImportTextEdits); name,
symbol,
priorWord,
completionList,
importSource,
undefined,
autoImportTextEdits
);
} }
} }
} }
@ -832,10 +884,9 @@ export class CompletionProvider {
// file, we can use that directory name as an implicit import target. // file, we can use that directory name as an implicit import target.
if (moduleSymbolMap.has(initPathPy) || moduleSymbolMap.has(initPathPyi)) { if (moduleSymbolMap.has(initPathPy) || moduleSymbolMap.has(initPathPyi)) {
const name = getFileName(fileDir); const name = getFileName(fileDir);
const moduleNameAndType = this._getModuleNameAndTypeFromFilePath( const moduleNameAndType = this._getModuleNameAndTypeFromFilePath(getDirectoryPath(fileDir));
getDirectoryPath(fileDir));
if (moduleNameAndType.moduleName) { if (moduleNameAndType.moduleName) {
const autoImportText = `Auto-import from ${ moduleNameAndType.moduleName }`; const autoImportText = `Auto-import from ${moduleNameAndType.moduleName}`;
const isDuplicateEntry = completionList.items.find(item => { const isDuplicateEntry = completionList.items.find(item => {
if (item.label === name) { if (item.label === name) {
@ -855,10 +906,23 @@ export class CompletionProvider {
if (!isDuplicateEntry) { if (!isDuplicateEntry) {
const autoImportTextEdits = this._getTextEditsForAutoImportByFilePath( const autoImportTextEdits = this._getTextEditsForAutoImportByFilePath(
name, importStatements, filePath, moduleNameAndType.moduleName, name,
moduleNameAndType ? moduleNameAndType.importType : ImportType.Local); importStatements,
this._addNameToCompletionList(name, CompletionItemKind.Module, priorWord, completionList, filePath,
name, '', autoImportText, undefined, autoImportTextEdits); moduleNameAndType.moduleName,
moduleNameAndType ? moduleNameAndType.importType : ImportType.Local
);
this._addNameToCompletionList(
name,
CompletionItemKind.Module,
priorWord,
completionList,
name,
'',
autoImportText,
undefined,
autoImportTextEdits
);
} }
} }
} }
@ -871,28 +935,36 @@ export class CompletionProvider {
// 'import from' statement. // 'import from' statement.
private _getModuleNameAndTypeFromFilePath(filePath: string): ModuleNameAndType { private _getModuleNameAndTypeFromFilePath(filePath: string): ModuleNameAndType {
const execEnvironment = this._configOptions.findExecEnvironment(this._filePath); const execEnvironment = this._configOptions.findExecEnvironment(this._filePath);
return this._importResolver.getModuleNameForImport( return this._importResolver.getModuleNameForImport(filePath, execEnvironment);
filePath, execEnvironment);
} }
private _getTextEditsForAutoImportByFilePath(symbolName: string, private _getTextEditsForAutoImportByFilePath(
importStatements: ImportStatementUtils.ImportStatements, filePath: string, symbolName: string,
moduleName: string, importType: ImportType): TextEditAction[] { importStatements: ImportStatementUtils.ImportStatements,
filePath: string,
moduleName: string,
importType: ImportType
): TextEditAction[] {
// Does an 'import from' statement already exist? If so, we'll reuse it. // Does an 'import from' statement already exist? If so, we'll reuse it.
const importStatement = importStatements.mapByFilePath.get(filePath); const importStatement = importStatements.mapByFilePath.get(filePath);
if (importStatement && importStatement.node.nodeType === ParseNodeType.ImportFrom) { if (importStatement && importStatement.node.nodeType === ParseNodeType.ImportFrom) {
return ImportStatementUtils.getTextEditsForAutoImportSymbolAddition( return ImportStatementUtils.getTextEditsForAutoImportSymbolAddition(
symbolName, importStatement, this._parseResults); symbolName,
importStatement,
this._parseResults
);
} }
return ImportStatementUtils.getTextEditsForAutoImportInsertion(symbolName, return ImportStatementUtils.getTextEditsForAutoImportInsertion(
importStatements, moduleName, importType, this._parseResults); symbolName,
importStatements,
moduleName,
importType,
this._parseResults
);
} }
private _getImportFromCompletions(importFromNode: ImportFromNode, private _getImportFromCompletions(importFromNode: ImportFromNode, priorWord: string): CompletionList | undefined {
priorWord: string): CompletionList | undefined {
// Don't attempt to provide completions for "from X import *". // Don't attempt to provide completions for "from X import *".
if (importFromNode.isWildcardImport) { if (importFromNode.isWildcardImport) {
return undefined; return undefined;
@ -907,25 +979,26 @@ export class CompletionProvider {
const completionList = CompletionList.create(); const completionList = CompletionList.create();
const resolvedPath = importInfo.resolvedPaths.length > 0 ? const resolvedPath =
importInfo.resolvedPaths[importInfo.resolvedPaths.length - 1] : ''; importInfo.resolvedPaths.length > 0 ? importInfo.resolvedPaths[importInfo.resolvedPaths.length - 1] : '';
const lookupResults = this._importLookup(resolvedPath); const lookupResults = this._importLookup(resolvedPath);
if (lookupResults) { if (lookupResults) {
this._addSymbolsForSymbolTable(lookupResults.symbolTable, this._addSymbolsForSymbolTable(
lookupResults.symbolTable,
name => { name => {
// Don't suggest symbols that have already been imported. // Don't suggest symbols that have already been imported.
return !importFromNode.imports.find( return !importFromNode.imports.find(imp => imp.name.value === name);
imp => imp.name.value === name);
}, },
priorWord, completionList); priorWord,
completionList
);
} }
// Add the implicit imports. // Add the implicit imports.
importInfo.implicitImports.forEach(implImport => { importInfo.implicitImports.forEach(implImport => {
if (!importFromNode.imports.find(imp => imp.name.value === implImport.name)) { if (!importFromNode.imports.find(imp => imp.name.value === implImport.name)) {
this._addNameToCompletionList(implImport.name, CompletionItemKind.Module, this._addNameToCompletionList(implImport.name, CompletionItemKind.Module, priorWord, completionList);
priorWord, completionList);
} }
}); });
@ -997,8 +1070,7 @@ export class CompletionProvider {
let scope = AnalyzerNodeInfo.getScope(curNode); let scope = AnalyzerNodeInfo.getScope(curNode);
if (scope) { if (scope) {
while (scope) { while (scope) {
this._addSymbolsForSymbolTable(scope.symbolTable, this._addSymbolsForSymbolTable(scope.symbolTable, () => true, priorWord, completionList);
() => true, priorWord, completionList);
scope = scope.parent; scope = scope.parent;
} }
break; break;
@ -1008,10 +1080,12 @@ export class CompletionProvider {
} }
} }
private _addSymbolsForSymbolTable(symbolTable: SymbolTable, private _addSymbolsForSymbolTable(
includeSymbolCallback: (name: string) => boolean, priorWord: string, symbolTable: SymbolTable,
completionList: CompletionList) { includeSymbolCallback: (name: string) => boolean,
priorWord: string,
completionList: CompletionList
) {
symbolTable.forEach((symbol, name) => { symbolTable.forEach((symbol, name) => {
// If there are no declarations or the symbol is not // If there are no declarations or the symbol is not
// exported from this scope, don't include it in the // exported from this scope, don't include it in the
@ -1022,9 +1096,15 @@ export class CompletionProvider {
}); });
} }
private _addSymbol(name: string, symbol: Symbol, priorWord: string, completionList: CompletionList, private _addSymbol(
autoImportSource?: string, textEdit?: TextEdit, additionalTextEdits?: TextEditAction[]) { name: string,
symbol: Symbol,
priorWord: string,
completionList: CompletionList,
autoImportSource?: string,
textEdit?: TextEdit,
additionalTextEdits?: TextEditAction[]
) {
let primaryDecl = getLastTypedDeclaredForSymbol(symbol); let primaryDecl = getLastTypedDeclaredForSymbol(symbol);
if (!primaryDecl) { if (!primaryDecl) {
const declarations = symbol.getDeclarations(); const declarations = symbol.getDeclarations();
@ -1063,8 +1143,9 @@ export class CompletionProvider {
case DeclarationType.Function: case DeclarationType.Function:
if (type.category === TypeCategory.OverloadedFunction) { if (type.category === TypeCategory.OverloadedFunction) {
typeDetail = type.overloads.map(overload => typeDetail = type.overloads
name + this._evaluator.printType(overload)).join('\n'); .map(overload => name + this._evaluator.printType(overload))
.join('\n');
} else { } else {
typeDetail = name + ': ' + this._evaluator.printType(type); typeDetail = name + ': ' + this._evaluator.printType(type);
} }
@ -1121,29 +1202,54 @@ export class CompletionProvider {
let autoImportText: string | undefined; let autoImportText: string | undefined;
if (autoImportSource) { if (autoImportSource) {
autoImportText = `Auto-import from ${ autoImportSource }`; autoImportText = `Auto-import from ${autoImportSource}`;
} }
this._addNameToCompletionList(name, itemKind, priorWord, completionList, this._addNameToCompletionList(
undefined, undefined, autoImportText, textEdit, name,
additionalTextEdits, symbol.id); itemKind,
priorWord,
completionList,
undefined,
undefined,
autoImportText,
textEdit,
additionalTextEdits,
symbol.id
);
} else { } else {
// Does the symbol have no declaration but instead has a synthesized type? // Does the symbol have no declaration but instead has a synthesized type?
const synthesizedType = symbol.getSynthesizedType(); const synthesizedType = symbol.getSynthesizedType();
if (synthesizedType) { if (synthesizedType) {
const itemKind: CompletionItemKind = CompletionItemKind.Variable; const itemKind: CompletionItemKind = CompletionItemKind.Variable;
this._addNameToCompletionList(name, itemKind, priorWord, completionList, this._addNameToCompletionList(
undefined, undefined, undefined, textEdit, name,
additionalTextEdits, symbol.id); itemKind,
priorWord,
completionList,
undefined,
undefined,
undefined,
textEdit,
additionalTextEdits,
symbol.id
);
} }
} }
} }
private _addNameToCompletionList(name: string, itemKind: CompletionItemKind, private _addNameToCompletionList(
filter: string, completionList: CompletionList, typeDetail?: string, name: string,
documentation?: string, autoImportText?: string, textEdit?: TextEdit, itemKind: CompletionItemKind,
additionalTextEdits?: TextEditAction[], symbolId?: number) { filter: string,
completionList: CompletionList,
typeDetail?: string,
documentation?: string,
autoImportText?: string,
textEdit?: TextEdit,
additionalTextEdits?: TextEditAction[],
symbolId?: number
) {
const similarity = StringUtils.computeCompletionSimilarity(filter, name); const similarity = StringUtils.computeCompletionSimilarity(filter, name);
if (similarity > similarityLimit) { if (similarity > similarityLimit) {
@ -1159,22 +1265,18 @@ export class CompletionProvider {
if (autoImportText) { if (autoImportText) {
// Force auto-import entries to the end. // Force auto-import entries to the end.
completionItem.sortText = completionItem.sortText = this._makeSortText(SortCategory.AutoImport, name, autoImportText);
this._makeSortText(SortCategory.AutoImport, name, autoImportText);
completionItemData.autoImportText = autoImportText; completionItemData.autoImportText = autoImportText;
} else if (SymbolNameUtils.isDunderName(name)) { } else if (SymbolNameUtils.isDunderName(name)) {
// Force dunder-named symbols to appear after all other symbols. // Force dunder-named symbols to appear after all other symbols.
completionItem.sortText = completionItem.sortText = this._makeSortText(SortCategory.DunderSymbol, name);
this._makeSortText(SortCategory.DunderSymbol, name); } else if (filter === '' && SymbolNameUtils.isPrivateOrProtectedName(name)) {
} else if (filter === '' && (SymbolNameUtils.isPrivateOrProtectedName(name))) {
// Distinguish between normal and private symbols only if there is // Distinguish between normal and private symbols only if there is
// currently no filter text. Once we get a single character to filter // currently no filter text. Once we get a single character to filter
// upon, we'll no longer differentiate. // upon, we'll no longer differentiate.
completionItem.sortText = completionItem.sortText = this._makeSortText(SortCategory.PrivateSymbol, name);
this._makeSortText(SortCategory.PrivateSymbol, name);
} else { } else {
completionItem.sortText = completionItem.sortText = this._makeSortText(SortCategory.NormalSymbol, name);
this._makeSortText(SortCategory.NormalSymbol, name);
} }
if (symbolId !== undefined) { if (symbolId !== undefined) {
@ -1228,13 +1330,11 @@ export class CompletionProvider {
private _getRecentListIndex(name: string, autoImportText: string) { private _getRecentListIndex(name: string, autoImportText: string) {
return CompletionProvider._mostRecentCompletions.findIndex( return CompletionProvider._mostRecentCompletions.findIndex(
item => item.label === name && item => item.label === name && item.autoImportText === autoImportText
item.autoImportText === autoImportText); );
} }
private _makeSortText(sortCategory: SortCategory, name: string, private _makeSortText(sortCategory: SortCategory, name: string, autoImportText = ''): string {
autoImportText = ''): string {
const recentListIndex = this._getRecentListIndex(name, autoImportText); const recentListIndex = this._getRecentListIndex(name, autoImportText);
// If the label is in the recent list, modify the category // If the label is in the recent list, modify the category
@ -1244,10 +1344,12 @@ export class CompletionProvider {
sortCategory = SortCategory.RecentAutoImport; sortCategory = SortCategory.RecentAutoImport;
} else if (sortCategory === SortCategory.ImportModuleName) { } else if (sortCategory === SortCategory.ImportModuleName) {
sortCategory = SortCategory.RecentImportModuleName; sortCategory = SortCategory.RecentImportModuleName;
} else if (sortCategory === SortCategory.Keyword || } else if (
sortCategory === SortCategory.NormalSymbol || sortCategory === SortCategory.Keyword ||
sortCategory === SortCategory.PrivateSymbol || sortCategory === SortCategory.NormalSymbol ||
sortCategory === SortCategory.DunderSymbol) { sortCategory === SortCategory.PrivateSymbol ||
sortCategory === SortCategory.DunderSymbol
) {
sortCategory = SortCategory.RecentKeywordOrSymbol; sortCategory = SortCategory.RecentKeywordOrSymbol;
} }
} }
@ -1256,9 +1358,7 @@ export class CompletionProvider {
// XX.YYYY.name // XX.YYYY.name
// where XX is the sort category // where XX is the sort category
// and YYYY is the index of the item in the MRU list // and YYYY is the index of the item in the MRU list
return this._formatInteger(sortCategory, 2) + '.' + return this._formatInteger(sortCategory, 2) + '.' + this._formatInteger(recentListIndex, 4) + '.' + name;
this._formatInteger(recentListIndex, 4) + '.' +
name;
} }
private _formatInteger(val: number, digits: number): string { private _formatInteger(val: number, digits: number): string {
@ -1292,13 +1392,12 @@ export class CompletionProvider {
return CompletionItemKind.Variable; return CompletionItemKind.Variable;
case DeclarationType.Variable: case DeclarationType.Variable:
return resolvedDeclaration.isConstant || resolvedDeclaration.isFinal ? return resolvedDeclaration.isConstant || resolvedDeclaration.isFinal
CompletionItemKind.Constant : ? CompletionItemKind.Constant
CompletionItemKind.Variable; : CompletionItemKind.Variable;
case DeclarationType.Function: case DeclarationType.Function:
return resolvedDeclaration.isMethod ? return resolvedDeclaration.isMethod ? CompletionItemKind.Method : CompletionItemKind.Function;
CompletionItemKind.Method : CompletionItemKind.Function;
case DeclarationType.Class: case DeclarationType.Class:
case DeclarationType.SpecialBuiltInClass: case DeclarationType.SpecialBuiltInClass:
@ -1318,30 +1417,35 @@ export class CompletionProvider {
importedSymbols: [] importedSymbols: []
}; };
const completions = this._importResolver.getCompletionSuggestions(this._filePath, const completions = this._importResolver.getCompletionSuggestions(
execEnvironment, moduleDescriptor, similarityLimit); this._filePath,
execEnvironment,
moduleDescriptor,
similarityLimit
);
const completionList = CompletionList.create(); const completionList = CompletionList.create();
// If we're in the middle of a "from X import Y" statement, offer // If we're in the middle of a "from X import Y" statement, offer
// the "import" keyword as a completion. // the "import" keyword as a completion.
if (!node.hasTrailingDot && node.parent && node.parent.nodeType === ParseNodeType.ImportFrom && if (
node.parent.missingImportKeyword) { !node.hasTrailingDot &&
node.parent &&
node.parent.nodeType === ParseNodeType.ImportFrom &&
node.parent.missingImportKeyword
) {
const keyword = 'import'; const keyword = 'import';
const completionItem = CompletionItem.create(keyword); const completionItem = CompletionItem.create(keyword);
completionItem.kind = CompletionItemKind.Keyword; completionItem.kind = CompletionItemKind.Keyword;
completionList.items.push(completionItem); completionList.items.push(completionItem);
completionItem.sortText = completionItem.sortText = this._makeSortText(SortCategory.Keyword, keyword);
this._makeSortText(SortCategory.Keyword, keyword);
} }
completions.forEach(completionName => { completions.forEach(completionName => {
const completionItem = CompletionItem.create(completionName); const completionItem = CompletionItem.create(completionName);
completionItem.kind = CompletionItemKind.Module; completionItem.kind = CompletionItemKind.Module;
completionList.items.push(completionItem); completionList.items.push(completionItem);
completionItem.sortText = completionItem.sortText = this._makeSortText(SortCategory.ImportModuleName, completionName);
this._makeSortText(SortCategory.ImportModuleName, completionName);
}); });
return completionList; return completionList;

28
server/src/languageService/definitionProvider.ts
View File

@ -1,14 +1,14 @@
/* /*
* definitionProvider.ts * definitionProvider.ts
* Copyright (c) Microsoft Corporation. * Copyright (c) Microsoft Corporation.
* Licensed under the MIT license. * Licensed under the MIT license.
* Author: Eric Traut * Author: Eric Traut
* *
* Logic that maps a position within a Python program file into * Logic that maps a position within a Python program file into
* a "definition" of the item that is referred to at that position. * a "definition" of the item that is referred to at that position.
* For example, if the location is within an import name, the * For example, if the location is within an import name, the
* definition is the top of the resolved import file. * definition is the top of the resolved import file.
*/ */
import * as ParseTreeUtils from '../analyzer/parseTreeUtils'; import * as ParseTreeUtils from '../analyzer/parseTreeUtils';
import { TypeEvaluator } from '../analyzer/typeEvaluator'; import { TypeEvaluator } from '../analyzer/typeEvaluator';
@ -18,9 +18,11 @@ import { ParseNodeType } from '../parser/parseNodes';
import { ParseResults } from '../parser/parser'; import { ParseResults } from '../parser/parser';
export class DefinitionProvider { export class DefinitionProvider {
static getDefinitionsForPosition(parseResults: ParseResults, position: Position, static getDefinitionsForPosition(
evaluator: TypeEvaluator): DocumentRange[] | undefined { parseResults: ParseResults,
position: Position,
evaluator: TypeEvaluator
): DocumentRange[] | undefined {
const offset = convertPositionToOffset(position, parseResults.tokenizerOutput.lines); const offset = convertPositionToOffset(position, parseResults.tokenizerOutput.lines);
if (offset === undefined) { if (offset === undefined) {
return undefined; return undefined;

85
server/src/languageService/documentSymbolProvider.ts
View File

@ -1,12 +1,12 @@
/* /*
* documentSymbolProvider.ts * documentSymbolProvider.ts
* Copyright (c) Microsoft Corporation. * Copyright (c) Microsoft Corporation.
* Licensed under the MIT license. * Licensed under the MIT license.
* Author: Eric Traut * Author: Eric Traut
* *
* Logic that enumerates all of the symbols within a specified * Logic that enumerates all of the symbols within a specified
* source file document. * source file document.
*/ */
import { DocumentSymbol, Location, SymbolInformation, SymbolKind } from 'vscode-languageserver'; import { DocumentSymbol, Location, SymbolInformation, SymbolKind } from 'vscode-languageserver';
import { URI } from 'vscode-uri'; import { URI } from 'vscode-uri';
@ -34,9 +34,13 @@ class FindSymbolTreeWalker extends ParseTreeWalker {
private _query: string | undefined; private _query: string | undefined;
private _evaluator: TypeEvaluator; private _evaluator: TypeEvaluator;
constructor(filePath: string, parseResults: ParseResults, symbolInfoResults: SymbolInformation[], constructor(
query: string | undefined, evaluator: TypeEvaluator) { filePath: string,
parseResults: ParseResults,
symbolInfoResults: SymbolInformation[],
query: string | undefined,
evaluator: TypeEvaluator
) {
super(); super();
this._filePath = filePath; this._filePath = filePath;
this._parseResults = parseResults; this._parseResults = parseResults;
@ -100,9 +104,7 @@ class FindSymbolTreeWalker extends ParseTreeWalker {
}); });
} }
private _addSymbolInformationFromDeclaration(name: string, declaration: Declaration, private _addSymbolInformationFromDeclaration(name: string, declaration: Declaration, containerName?: string) {
containerName?: string) {
if (declaration.path !== this._filePath) { if (declaration.path !== this._filePath) {
return; return;
} }
@ -179,8 +181,7 @@ function getSymbolKind(name: string, declaration: Declaration, evaluator: TypeEv
if (name === '_') { if (name === '_') {
return; return;
} }
symbolKind = declaration.isConstant || declaration.isFinal ? symbolKind = declaration.isConstant || declaration.isFinal ? SymbolKind.Constant : SymbolKind.Variable;
SymbolKind.Constant : SymbolKind.Variable;
break; break;
default: default:
@ -191,10 +192,12 @@ function getSymbolKind(name: string, declaration: Declaration, evaluator: TypeEv
return symbolKind; return symbolKind;
} }
function getDocumentSymbolsRecursive(node: AnalyzerNodeInfo.ScopedNode, function getDocumentSymbolsRecursive(
docSymbolResults: DocumentSymbol[], parseResults: ParseResults, node: AnalyzerNodeInfo.ScopedNode,
evaluator: TypeEvaluator) { docSymbolResults: DocumentSymbol[],
parseResults: ParseResults,
evaluator: TypeEvaluator
) {
const scope = AnalyzerNodeInfo.getScope(node); const scope = AnalyzerNodeInfo.getScope(node);
if (!scope) { if (!scope) {
return; return;
@ -218,10 +221,13 @@ function getDocumentSymbolsRecursive(node: AnalyzerNodeInfo.ScopedNode,
}); });
} }
function getDocumentSymbolRecursive(name: string, declaration: Declaration, function getDocumentSymbolRecursive(
evaluator: TypeEvaluator, parseResults: ParseResults, name: string,
docSymbolResults: DocumentSymbol[]) { declaration: Declaration,
evaluator: TypeEvaluator,
parseResults: ParseResults,
docSymbolResults: DocumentSymbol[]
) {
if (declaration.type === DeclarationType.Alias) { if (declaration.type === DeclarationType.Alias) {
return; return;
} }
@ -235,14 +241,14 @@ function getDocumentSymbolRecursive(name: string, declaration: Declaration,
let range = selectionRange; let range = selectionRange;
const children: DocumentSymbol[] = []; const children: DocumentSymbol[] = [];
if (declaration.type === DeclarationType.Class || if (declaration.type === DeclarationType.Class || declaration.type === DeclarationType.Function) {
declaration.type === DeclarationType.Function) {
getDocumentSymbolsRecursive(declaration.node, children, parseResults, evaluator); getDocumentSymbolsRecursive(declaration.node, children, parseResults, evaluator);
const nameRange = convertOffsetsToRange(declaration.node.start, const nameRange = convertOffsetsToRange(
declaration.node.start,
declaration.node.name.start + declaration.node.length, declaration.node.name.start + declaration.node.length,
parseResults.tokenizerOutput.lines); parseResults.tokenizerOutput.lines
);
range = nameRange; range = nameRange;
} }
@ -258,17 +264,22 @@ function getDocumentSymbolRecursive(name: string, declaration: Declaration,
} }
export class DocumentSymbolProvider { export class DocumentSymbolProvider {
static addSymbolsForDocument(symbolList: SymbolInformation[], query: string | undefined, static addSymbolsForDocument(
filePath: string, parseResults: ParseResults, evaluator: TypeEvaluator) { symbolList: SymbolInformation[],
query: string | undefined,
const symbolTreeWalker = new FindSymbolTreeWalker(filePath, parseResults, filePath: string,
symbolList, query, evaluator); parseResults: ParseResults,
evaluator: TypeEvaluator
) {
const symbolTreeWalker = new FindSymbolTreeWalker(filePath, parseResults, symbolList, query, evaluator);
symbolTreeWalker.findSymbols(); symbolTreeWalker.findSymbols();
} }
static addHierarchicalSymbolsForDocument(symbolList: DocumentSymbol[], static addHierarchicalSymbolsForDocument(
parseResults: ParseResults, evaluator: TypeEvaluator) { symbolList: DocumentSymbol[],
parseResults: ParseResults,
evaluator: TypeEvaluator
) {
getDocumentSymbolsRecursive(parseResults.parseTree, symbolList, parseResults, evaluator); getDocumentSymbolsRecursive(parseResults.parseTree, symbolList, parseResults, evaluator);
} }
} }

56
server/src/languageService/hoverProvider.ts
View File

@ -1,13 +1,13 @@
/* /*
* hoverProvider.ts * hoverProvider.ts
* Copyright (c) Microsoft Corporation. * Copyright (c) Microsoft Corporation.
* Licensed under the MIT license. * Licensed under the MIT license.
* Author: Eric Traut * Author: Eric Traut
* *
* Logic that maps a position within a Python program file into * Logic that maps a position within a Python program file into
* markdown text that is displayed when the user hovers over that * markdown text that is displayed when the user hovers over that
* position within a smart editor. * position within a smart editor.
*/ */
import { Declaration, DeclarationType } from '../analyzer/declaration'; import { Declaration, DeclarationType } from '../analyzer/declaration';
import { convertDocStringToMarkdown } from '../analyzer/docStringUtils'; import { convertDocStringToMarkdown } from '../analyzer/docStringUtils';
@ -32,9 +32,11 @@ export interface HoverResults {
} }
export class HoverProvider { export class HoverProvider {
static getHoverForPosition(parseResults: ParseResults, position: Position, static getHoverForPosition(
evaluator: TypeEvaluator): HoverResults | undefined { parseResults: ParseResults,
position: Position,
evaluator: TypeEvaluator
): HoverResults | undefined {
const offset = convertPositionToOffset(position, parseResults.tokenizerOutput.lines); const offset = convertPositionToOffset(position, parseResults.tokenizerOutput.lines);
if (offset === undefined) { if (offset === undefined) {
return undefined; return undefined;
@ -84,36 +86,34 @@ export class HoverProvider {
return results.parts.length > 0 ? results : undefined; return results.parts.length > 0 ? results : undefined;
} }
private static _addResultsForDeclaration(parts: HoverTextPart[], declaration: Declaration, private static _addResultsForDeclaration(
node: NameNode, evaluator: TypeEvaluator): void { parts: HoverTextPart[],
declaration: Declaration,
node: NameNode,
evaluator: TypeEvaluator
): void {
const resolvedDecl = evaluator.resolveAliasDeclaration(declaration); const resolvedDecl = evaluator.resolveAliasDeclaration(declaration);
if (!resolvedDecl) { if (!resolvedDecl) {
this._addResultsPart(parts, `(import) ` + node.value + this._addResultsPart(parts, `(import) ` + node.value + this._getTypeText(node, evaluator), true);
this._getTypeText(node, evaluator), true);
return; return;
} }
switch (resolvedDecl.type) { switch (resolvedDecl.type) {
case DeclarationType.Intrinsic: { case DeclarationType.Intrinsic: {
this._addResultsPart(parts, node.value + this._addResultsPart(parts, node.value + this._getTypeText(node, evaluator), true);
this._getTypeText(node, evaluator), true);
this._addDocumentationPart(parts, node, evaluator); this._addDocumentationPart(parts, node, evaluator);
break; break;
} }
case DeclarationType.Variable: { case DeclarationType.Variable: {
const label = resolvedDecl.isConstant || resolvedDecl.isFinal ? const label = resolvedDecl.isConstant || resolvedDecl.isFinal ? 'constant' : 'variable';
'constant' : 'variable'; this._addResultsPart(parts, `(${label}) ` + node.value + this._getTypeText(node, evaluator), true);
this._addResultsPart(parts, `(${ label }) ` + node.value +
this._getTypeText(node, evaluator), true);
this._addDocumentationPart(parts, node, evaluator); this._addDocumentationPart(parts, node, evaluator);
break; break;
} }
case DeclarationType.Parameter: { case DeclarationType.Parameter: {
this._addResultsPart(parts, '(parameter) ' + node.value + this._addResultsPart(parts, '(parameter) ' + node.value + this._getTypeText(node, evaluator), true);
this._getTypeText(node, evaluator), true);
this._addDocumentationPart(parts, node, evaluator); this._addDocumentationPart(parts, node, evaluator);
break; break;
} }
@ -129,12 +129,10 @@ export class HoverProvider {
let label = 'function'; let label = 'function';
if (resolvedDecl.isMethod) { if (resolvedDecl.isMethod) {
const declaredType = evaluator.getTypeForDeclaration(resolvedDecl); const declaredType = evaluator.getTypeForDeclaration(resolvedDecl);
label = declaredType && isProperty(declaredType) ? label = declaredType && isProperty(declaredType) ? 'property' : 'method';
'property' : 'method';
} }
this._addResultsPart(parts, `(${ label }) ` + node.value + this._addResultsPart(parts, `(${label}) ` + node.value + this._getTypeText(node, evaluator), true);
this._getTypeText(node, evaluator), true);
this._addDocumentationPart(parts, node, evaluator); this._addDocumentationPart(parts, node, evaluator);
break; break;
} }

89
server/src/languageService/importSorter.ts
View File

@ -1,12 +1,12 @@
/* /*
* importSorter.ts * importSorter.ts
* Copyright (c) Microsoft Corporation. * Copyright (c) Microsoft Corporation.
* Licensed under the MIT license. * Licensed under the MIT license.
* Author: Eric Traut * Author: Eric Traut
* *
* Provides code that sorts and formats import statements within a * Provides code that sorts and formats import statements within a
* python source file. * python source file.
*/ */
import { ImportType } from '../analyzer/importResult'; import { ImportType } from '../analyzer/importResult';
import * as ImportStatementUtils from '../analyzer/importStatementUtils'; import * as ImportStatementUtils from '../analyzer/importStatementUtils';
@ -33,11 +33,11 @@ export class ImportSorter {
sort(): TextEditAction[] { sort(): TextEditAction[] {
const actions: TextEditAction[] = []; const actions: TextEditAction[] = [];
const importStatements = ImportStatementUtils.getTopLevelImports( const importStatements = ImportStatementUtils.getTopLevelImports(this._parseResults.parseTree);
this._parseResults.parseTree);
const sortedStatements = importStatements.orderedImports. const sortedStatements = importStatements.orderedImports
map(s => s).sort((a, b) => { .map(s => s)
.sort((a, b) => {
return this._compareImportStatements(a, b); return this._compareImportStatements(a, b);
}); });
@ -46,23 +46,19 @@ export class ImportSorter {
return []; return [];
} }
const primaryRange = this._getPrimaryReplacementRange( const primaryRange = this._getPrimaryReplacementRange(importStatements.orderedImports);
importStatements.orderedImports);
actions.push({ actions.push({
range: primaryRange, range: primaryRange,
replacementText: this._generateSortedImportText(sortedStatements) replacementText: this._generateSortedImportText(sortedStatements)
}); });
this._addSecondaryReplacementRanges( this._addSecondaryReplacementRanges(importStatements.orderedImports, actions);
importStatements.orderedImports, actions);
return actions; return actions;
} }
private _compareImportStatements(a: ImportStatementUtils.ImportStatement, private _compareImportStatements(a: ImportStatementUtils.ImportStatement, b: ImportStatementUtils.ImportStatement) {
b: ImportStatementUtils.ImportStatement) {
const aImportGroup = this._getImportGroup(a); const aImportGroup = this._getImportGroup(a);
const bImportGroup = this._getImportGroup(b); const bImportGroup = this._getImportGroup(b);
@ -72,16 +68,17 @@ export class ImportSorter {
return 1; return 1;
} }
return (a.moduleName < b.moduleName) ? -1 : 1; return a.moduleName < b.moduleName ? -1 : 1;
} }
private _getImportGroup(statement: ImportStatementUtils.ImportStatement): ImportGroup { private _getImportGroup(statement: ImportStatementUtils.ImportStatement): ImportGroup {
if (statement.importResult) { if (statement.importResult) {
if (statement.importResult.importType === ImportType.BuiltIn) { if (statement.importResult.importType === ImportType.BuiltIn) {
return ImportGroup.BuiltIn; return ImportGroup.BuiltIn;
} else if (statement.importResult.importType === ImportType.ThirdParty || } else if (
statement.importResult.isLocalTypingsFile) { statement.importResult.importType === ImportType.ThirdParty ||
statement.importResult.isLocalTypingsFile
) {
return ImportGroup.ThirdParty; return ImportGroup.ThirdParty;
} }
@ -98,9 +95,7 @@ export class ImportSorter {
// Determines the text range for the existing primary block of import statements. // Determines the text range for the existing primary block of import statements.
// If there are other blocks of import statements separated by other statements, // If there are other blocks of import statements separated by other statements,
// we'll ignore these other blocks for now. // we'll ignore these other blocks for now.
private _getPrimaryReplacementRange(statements: ImportStatementUtils.ImportStatement[]): private _getPrimaryReplacementRange(statements: ImportStatementUtils.ImportStatement[]): Range {
Range {
let statementLimit = statements.findIndex(s => s.followsNonImportStatement); let statementLimit = statements.findIndex(s => s.followsNonImportStatement);
if (statementLimit < 0) { if (statementLimit < 0) {
statementLimit = statements.length; statementLimit = statements.length;
@ -108,18 +103,17 @@ export class ImportSorter {
const lastStatement = statements[statementLimit - 1].node; const lastStatement = statements[statementLimit - 1].node;
return { return {
start: convertOffsetToPosition( start: convertOffsetToPosition(statements[0].node.start, this._parseResults.tokenizerOutput.lines),
statements[0].node.start, this._parseResults.tokenizerOutput.lines), end: convertOffsetToPosition(TextRange.getEnd(lastStatement), this._parseResults.tokenizerOutput.lines)
end: convertOffsetToPosition(
TextRange.getEnd(lastStatement), this._parseResults.tokenizerOutput.lines)
}; };
} }
// If import statements are separated by other statements, we will remove the old // If import statements are separated by other statements, we will remove the old
// secondary blocks. // secondary blocks.
private _addSecondaryReplacementRanges(statements: ImportStatementUtils.ImportStatement[], private _addSecondaryReplacementRanges(
actions: TextEditAction[]) { statements: ImportStatementUtils.ImportStatement[],
actions: TextEditAction[]
) {
let secondaryBlockStart = statements.findIndex(s => s.followsNonImportStatement); let secondaryBlockStart = statements.findIndex(s => s.followsNonImportStatement);
if (secondaryBlockStart < 0) { if (secondaryBlockStart < 0) {
return; return;
@ -127,7 +121,8 @@ export class ImportSorter {
while (true) { while (true) {
let secondaryBlockLimit = statements.findIndex( let secondaryBlockLimit = statements.findIndex(
(s, index) => index > secondaryBlockStart && s.followsNonImportStatement); (s, index) => index > secondaryBlockStart && s.followsNonImportStatement
);
if (secondaryBlockLimit < 0) { if (secondaryBlockLimit < 0) {
secondaryBlockLimit = statements.length; secondaryBlockLimit = statements.length;
} }
@ -136,10 +131,12 @@ export class ImportSorter {
range: { range: {
start: convertOffsetToPosition( start: convertOffsetToPosition(
statements[secondaryBlockStart].node.start, statements[secondaryBlockStart].node.start,
this._parseResults.tokenizerOutput.lines), this._parseResults.tokenizerOutput.lines
),
end: convertOffsetToPosition( end: convertOffsetToPosition(
TextRange.getEnd(statements[secondaryBlockLimit - 1].node), TextRange.getEnd(statements[secondaryBlockLimit - 1].node),
this._parseResults.tokenizerOutput.lines) this._parseResults.tokenizerOutput.lines
)
}, },
replacementText: '' replacementText: ''
}); });
@ -165,11 +162,9 @@ export class ImportSorter {
let importLine: string; let importLine: string;
if (statement.node.nodeType === ParseNodeType.Import) { if (statement.node.nodeType === ParseNodeType.Import) {
importLine = this._formatImportNode(statement.subnode!, importLine = this._formatImportNode(statement.subnode!, statement.moduleName);
statement.moduleName);
} else { } else {
importLine = this._formatImportFromNode(statement.node, importLine = this._formatImportFromNode(statement.node, statement.moduleName);
statement.moduleName);
} }
// If this isn't the last statement, add a newline. // If this isn't the last statement, add a newline.
@ -184,27 +179,27 @@ export class ImportSorter {
} }
private _formatImportNode(subnode: ImportAsNode, moduleName: string): string { private _formatImportNode(subnode: ImportAsNode, moduleName: string): string {
let importText = `import ${ moduleName }`; let importText = `import ${moduleName}`;
if (subnode.alias) { if (subnode.alias) {
importText += ` as ${ subnode.alias.value }`; importText += ` as ${subnode.alias.value}`;
} }
return importText; return importText;
} }
private _formatImportFromNode(node: ImportFromNode, moduleName: string): string { private _formatImportFromNode(node: ImportFromNode, moduleName: string): string {
const symbols = node.imports. const symbols = node.imports
sort((a, b) => this._compareSymbols(a, b)). .sort((a, b) => this._compareSymbols(a, b))
map(symbol => { .map(symbol => {
let symbolText = symbol.name.value; let symbolText = symbol.name.value;
if (symbol.alias) { if (symbol.alias) {
symbolText += ` as ${ symbol.alias.value }`; symbolText += ` as ${symbol.alias.value}`;
} }
return symbolText; return symbolText;
}); });
let cumulativeText = `from ${ moduleName } import `; let cumulativeText = `from ${moduleName} import `;
const symbolText = symbols.join(', '); const symbolText = symbols.join(', ');
if (cumulativeText.length + symbolText.length <= _maxLineLength) { if (cumulativeText.length + symbolText.length <= _maxLineLength) {
return cumulativeText + symbolText; return cumulativeText + symbolText;

46
server/src/languageService/quickActions.ts
View File

@ -1,11 +1,11 @@
/* /*
* quickActions.ts * quickActions.ts
* Copyright (c) Microsoft Corporation. * Copyright (c) Microsoft Corporation.
* Licensed under the MIT license. * Licensed under the MIT license.
* Author: Eric Traut * Author: Eric Traut
* *
* Provides support for miscellaneous quick actions. * Provides support for miscellaneous quick actions.
*/ */
import { ImportType } from '../analyzer/importResult'; import { ImportType } from '../analyzer/importResult';
import * as ImportStatementUtils from '../analyzer/importStatementUtils'; import * as ImportStatementUtils from '../analyzer/importStatementUtils';
@ -18,9 +18,7 @@ import { ParseNode, ParseNodeType } from '../parser/parseNodes';
import { ParseResults } from '../parser/parser'; import { ParseResults } from '../parser/parser';
import { ImportSorter } from './importSorter'; import { ImportSorter } from './importSorter';
export function performQuickAction(command: string, args: any[], export function performQuickAction(command: string, args: any[], parseResults: ParseResults) {
parseResults: ParseResults) {
if (command === Commands.orderImports) { if (command === Commands.orderImports) {
const importSorter = new ImportSorter(parseResults); const importSorter = new ImportSorter(parseResults);
return importSorter.sort(); return importSorter.sort();
@ -36,9 +34,7 @@ export function performQuickAction(command: string, args: any[],
// Inserts text into the document to wrap an existing type annotation // Inserts text into the document to wrap an existing type annotation
// with "Optional[X]". // with "Optional[X]".
function _addMissingOptionalToParam(parseResults: ParseResults, function _addMissingOptionalToParam(parseResults: ParseResults, offset: number): TextEditAction[] {
offset: number): TextEditAction[] {
let node: ParseNode | undefined = ParseTreeUtils.findNodeByOffset(parseResults.parseTree, offset); let node: ParseNode | undefined = ParseTreeUtils.findNodeByOffset(parseResults.parseTree, offset);
while (node) { while (node) {
if (node.nodeType === ParseNodeType.Parameter) { if (node.nodeType === ParseNodeType.Parameter) {
@ -54,10 +50,8 @@ function _addMissingOptionalToParam(parseResults: ParseResults,
const editActions: TextEditAction[] = []; const editActions: TextEditAction[] = [];
const startPos = convertOffsetToPosition( const startPos = convertOffsetToPosition(node.typeAnnotation.start, parseResults.tokenizerOutput.lines);
node.typeAnnotation.start, parseResults.tokenizerOutput.lines); const endPos = convertOffsetToPosition(TextRange.getEnd(node.typeAnnotation), parseResults.tokenizerOutput.lines);
const endPos = convertOffsetToPosition(
TextRange.getEnd(node.typeAnnotation), parseResults.tokenizerOutput.lines);
editActions.push({ editActions.push({
range: { start: startPos, end: startPos }, range: { start: startPos, end: startPos },
@ -69,19 +63,25 @@ function _addMissingOptionalToParam(parseResults: ParseResults,
}); });
// Add the import statement if necessary. // Add the import statement if necessary.
const importStatements = ImportStatementUtils.getTopLevelImports( const importStatements = ImportStatementUtils.getTopLevelImports(parseResults.parseTree);
parseResults.parseTree); const importStatement = importStatements.orderedImports.find(imp => imp.moduleName === 'typing');
const importStatement = importStatements.orderedImports.find(
imp => imp.moduleName === 'typing');
// If there's an existing import statement, insert into it. // If there's an existing import statement, insert into it.
if (importStatement && importStatement.node.nodeType === ParseNodeType.ImportFrom) { if (importStatement && importStatement.node.nodeType === ParseNodeType.ImportFrom) {
const additionalEditActions = ImportStatementUtils.getTextEditsForAutoImportSymbolAddition( const additionalEditActions = ImportStatementUtils.getTextEditsForAutoImportSymbolAddition(
'Optional', importStatement, parseResults); 'Optional',
importStatement,
parseResults
);
editActions.push(...additionalEditActions); editActions.push(...additionalEditActions);
} else { } else {
const additionalEditActions = ImportStatementUtils.getTextEditsForAutoImportInsertion( const additionalEditActions = ImportStatementUtils.getTextEditsForAutoImportInsertion(
'Optional', importStatements, 'typing', ImportType.BuiltIn, parseResults); 'Optional',
importStatements,
'typing',
ImportType.BuiltIn,
parseResults
);
editActions.push(...additionalEditActions); editActions.push(...additionalEditActions);
} }

80
server/src/languageService/referencesProvider.ts
View File

@ -1,12 +1,12 @@
/* /*
* referencesProvider.ts * referencesProvider.ts
* Copyright (c) Microsoft Corporation. * Copyright (c) Microsoft Corporation.
* Licensed under the MIT license. * Licensed under the MIT license.
* Author: Eric Traut * Author: Eric Traut
* *
* Logic that finds all of the references to a symbol specified * Logic that finds all of the references to a symbol specified
* by a location within a file. * by a location within a file.
*/ */
import * as AnalyzerNodeInfo from '../analyzer/analyzerNodeInfo'; import * as AnalyzerNodeInfo from '../analyzer/analyzerNodeInfo';
import { Declaration, DeclarationType } from '../analyzer/declaration'; import { Declaration, DeclarationType } from '../analyzer/declaration';
@ -15,7 +15,7 @@ import * as ParseTreeUtils from '../analyzer/parseTreeUtils';
import { ParseTreeWalker } from '../analyzer/parseTreeWalker'; import { ParseTreeWalker } from '../analyzer/parseTreeWalker';
import { TypeEvaluator } from '../analyzer/typeEvaluator'; import { TypeEvaluator } from '../analyzer/typeEvaluator';
import { convertOffsetToPosition, convertPositionToOffset } from '../common/positionUtils'; import { convertOffsetToPosition, convertPositionToOffset } from '../common/positionUtils';
import { DocumentRange,Position } from '../common/textRange'; import { DocumentRange, Position } from '../common/textRange';
import { TextRange } from '../common/textRange'; import { TextRange } from '../common/textRange';
import { NameNode, ParseNode, ParseNodeType } from '../parser/parseNodes'; import { NameNode, ParseNode, ParseNodeType } from '../parser/parseNodes';
import { ParseResults } from '../parser/parser'; import { ParseResults } from '../parser/parser';
@ -34,10 +34,13 @@ class FindReferencesTreeWalker extends ParseTreeWalker {
private _includeDeclaration: boolean; private _includeDeclaration: boolean;
private _evaluator: TypeEvaluator; private _evaluator: TypeEvaluator;
constructor(parseResults: ParseResults, filePath: string, constructor(
referencesResult: ReferencesResult, includeDeclaration: boolean, parseResults: ParseResults,
evaluator: TypeEvaluator) { filePath: string,
referencesResult: ReferencesResult,
includeDeclaration: boolean,
evaluator: TypeEvaluator
) {
super(); super();
this._parseResults = parseResults; this._parseResults = parseResults;
this._filePath = filePath; this._filePath = filePath;
@ -68,7 +71,10 @@ class FindReferencesTreeWalker extends ParseTreeWalker {
path: this._filePath, path: this._filePath,
range: { range: {
start: convertOffsetToPosition(node.start, this._parseResults.tokenizerOutput.lines), start: convertOffsetToPosition(node.start, this._parseResults.tokenizerOutput.lines),
end: convertOffsetToPosition(TextRange.getEnd(node), this._parseResults.tokenizerOutput.lines) end: convertOffsetToPosition(
TextRange.getEnd(node),
this._parseResults.tokenizerOutput.lines
)
} }
}); });
} }
@ -87,15 +93,19 @@ class FindReferencesTreeWalker extends ParseTreeWalker {
// The reference results declarations are already resolved, so we don't // The reference results declarations are already resolved, so we don't
// need to call resolveAliasDeclaration on them. // need to call resolveAliasDeclaration on them.
return this._referencesResult.declarations.some(decl => return this._referencesResult.declarations.some(decl =>
DeclarationUtils.areDeclarationsSame(decl, resolvedDecl)); DeclarationUtils.areDeclarationsSame(decl, resolvedDecl)
);
} }
} }
export class ReferencesProvider { export class ReferencesProvider {
static getReferencesForPosition(parseResults: ParseResults, filePath: string, static getReferencesForPosition(
position: Position, includeDeclaration: boolean, parseResults: ParseResults,
evaluator: TypeEvaluator): ReferencesResult | undefined { filePath: string,
position: Position,
includeDeclaration: boolean,
evaluator: TypeEvaluator
): ReferencesResult | undefined {
const offset = convertPositionToOffset(position, parseResults.tokenizerOutput.lines); const offset = convertPositionToOffset(position, parseResults.tokenizerOutput.lines);
if (offset === undefined) { if (offset === undefined) {
return undefined; return undefined;
@ -133,7 +143,8 @@ export class ReferencesProvider {
// Parameters are local to a scope, so they don't require a global search. // Parameters are local to a scope, so they don't require a global search.
// If it's a named argument referring to a parameter, we still need to perform // If it's a named argument referring to a parameter, we still need to perform
// the global search. // the global search.
const requiresGlobalSearch = symbolDeclType !== DeclarationType.Parameter || const requiresGlobalSearch =
symbolDeclType !== DeclarationType.Parameter ||
(node.parent !== undefined && node.parent.nodeType === ParseNodeType.Argument); (node.parent !== undefined && node.parent.nodeType === ParseNodeType.Argument);
const results: ReferencesResult = { const results: ReferencesResult = {
@ -143,19 +154,32 @@ export class ReferencesProvider {
locations: [] locations: []
}; };
const refTreeWalker = new FindReferencesTreeWalker(parseResults, const refTreeWalker = new FindReferencesTreeWalker(
filePath, results, includeDeclaration, evaluator); parseResults,
filePath,
results,
includeDeclaration,
evaluator
);
refTreeWalker.findReferences(); refTreeWalker.findReferences();
return results; return results;
} }
static addReferences(parseResults: ParseResults, filePath: string, static addReferences(
referencesResult: ReferencesResult, includeDeclaration: boolean, parseResults: ParseResults,
evaluator: TypeEvaluator): void { filePath: string,
referencesResult: ReferencesResult,
const refTreeWalker = new FindReferencesTreeWalker(parseResults, includeDeclaration: boolean,
filePath, referencesResult, includeDeclaration, evaluator); evaluator: TypeEvaluator
): void {
const refTreeWalker = new FindReferencesTreeWalker(
parseResults,
filePath,
referencesResult,
includeDeclaration,
evaluator
);
refTreeWalker.findReferences(); refTreeWalker.findReferences();
} }
} }

27
server/src/languageService/signatureHelpProvider.ts
View File

@ -1,13 +1,13 @@
/* /*
* signatureHelpProvider.ts * signatureHelpProvider.ts
* Copyright (c) Microsoft Corporation. * Copyright (c) Microsoft Corporation.
* Licensed under the MIT license. * Licensed under the MIT license.
* Author: Eric Traut * Author: Eric Traut
* *
* Logic that maps a position within a Python call node into info * Logic that maps a position within a Python call node into info
* that can be presented to the developer to help fill in the remaining * that can be presented to the developer to help fill in the remaining
* arguments for the call. * arguments for the call.
*/ */
import { extractParameterDocumentation } from '../analyzer/docStringUtils'; import { extractParameterDocumentation } from '../analyzer/docStringUtils';
import * as ParseTreeUtils from '../analyzer/parseTreeUtils'; import * as ParseTreeUtils from '../analyzer/parseTreeUtils';
@ -36,10 +36,11 @@ export interface SignatureHelpResults {
} }
export class SignatureHelpProvider { export class SignatureHelpProvider {
static getSignatureHelpForPosition(parseResults: ParseResults, position: Position, static getSignatureHelpForPosition(
evaluator: TypeEvaluator): parseResults: ParseResults,
SignatureHelpResults | undefined { position: Position,
evaluator: TypeEvaluator
): SignatureHelpResults | undefined {
const offset = convertPositionToOffset(position, parseResults.tokenizerOutput.lines); const offset = convertPositionToOffset(position, parseResults.tokenizerOutput.lines);
if (offset === undefined) { if (offset === undefined) {
return undefined; return undefined;

20
server/src/parser/characterStream.ts
View File

@ -1,14 +1,14 @@
/* /*
* characterStream.ts * characterStream.ts
* Copyright (c) Microsoft Corporation. * Copyright (c) Microsoft Corporation.
* Licensed under the MIT license. * Licensed under the MIT license.
* Author: Eric Traut * Author: Eric Traut
* *
* Based on code from vscode-python repository: * Based on code from vscode-python repository:
* https://github.com/Microsoft/vscode-python * https://github.com/Microsoft/vscode-python
* *
* Class that represents a stream of characters. * Class that represents a stream of characters.
*/ */
import Char from 'typescript-char'; import Char from 'typescript-char';

Some files were not shown because too many files have changed in this diff Show More