microsoft/pyright
1
1
Fork 0
mirror of https://github.com/microsoft/pyright.git synced 2024-10-26 02:38:31 +03:00
Code Issues Projects Releases Wiki Activity

ported typescript fourslash test framework to pyright

Browse Source 
this PR consists of 3 main changes.

refactoring on pyright, 2. bunch of helper methods in src/common, 3. fourslash related codes in src/test
...

for 1. refactoring.

it has 2 refactorings on pyright and that affects many pyright code. which made file counts of this PR high.

first refactoring is rename of DiagnosticTextPosition and related types to Position/Range and move them out from diagnostics.ts to textRange.ts.

this is done since I needed line column and it is not diagnostic specific concept.

second one is introducing virtual file system to pyright (server/src/common/vfs.ts).

this is needed since I needed pyright to run over virtual file system constructed from fourslash files.
also, after this, nobody in pyright should interact with file system directly (no import * from "fs").

...

for 2. bunch of helper methods in src/common

it is just bunch of helper methods over path, collections, strings and etc fourslash test framework was using. mostly about string, path comparisons, adding/removing collections without caring about corner cases and etc.

...

for 3. fourslash related codes in src/test

all code related to it should be under src/test/harness and actual tests will be in src/test/fourslash

fourslash test framework is consist of 5 components

virtual file system - this is a file system (server/src/tests/harness/vfs/filesystem.ts) emulating real one. it is a standalone lib people can use for other purpose as well. see (server/src/tests/filesystem.test.ts) on how to use it.

fourslash test reader/parser
it reads fourslash file like (server/src/tests/fourslash/dataclass1.fourslash.ts) or (server/src/tests/fourSlashParser.test.ts) and return FourSlashData that contains source + markup information.
this is also a standalone library one can use on other scenarios just for markups.

testState
it maintain current test states. such as what file is opened, where the caret position is, what is current active files, selection is. and hold pyright program, virtual file system and etc. this will be given to a fourslash test file when the test run. and people can interact with it to manipulate or verify test results. see (server/src/tests/testState.test.ts) for what it currently supports. this should keep updated as we add new tests and new test ability.
one thing to remember is that, TestState is what run at runtime (implementation), but for editting, fourslash.ts (declaration) (server/src/tests/fourslash/fourslash.ts) is the one that is used.
that is why there is reference path="fourslash.ts" at the top of each fourslash test file (ex, server/src/tests/fourslash/dataclass1.fourslash.ts). because of that, it is important to put declaration in fourslash.ts if one added new functionality in TestState.

fourslash test runner
this puts everything together and run given fourslash markup strings. to learn how it actually run code. see (server/src/tests/harness/fourslash/runner.ts).
basically, it gets sources (python code) out of fourslash string, and construct virtual file system and connect it to pyright using TestState and then compile the given fourslash markup string since it is actually pure typescript code and then wrapped the compiled javascript code under function and eval the code and run it with TestState as argument. the reference path=".." trick works since it gets ignored within method body as a regular comment.
this is also a standalone lib people can use in other way if wants. see (

server/src/tests/testState.test.ts

test('VerifyDiagnosticsTest1', () => {
) on how to use it.

jest test suite
this connects fourslash test framework to jest. it basically read fourslash test files in **/test/fourslahs/*.fourslash.ts and register it to jest and run them with the fourslash test runner.
(server/src/tests/fourSlashRunner.test.ts)
...

PR also includes some misc changes such as ignoring eslint on fourslash.ts files. or js/ts formatting rule such as putting space after template string since that seems what pyright currently has as a code style. and this also includes some of checker.test.ts converted to fourslahst test to show differences. and debugger launch settings for fourslash tests.

...

how to run fourslash tests.

on command line, if one runs jest tests (ex, npm run test), it will run fourslash tests as well. in vscode, if users choose vscode-jest-tests, it will run fourslash tests as well. if user opens fourslash.ts file and then select fourslash current file in debugger configuration and F5, vscode will run only that fourslash test file.
This commit is contained in:
HeeJae Chang 2020-02-05 13:25:40 -08:00
parent fc8f190c6c
commit 59c693b65a
75 changed files with 6979 additions and 837 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
.eslintignore Normal file
View File

@ -0,0 +1 @@
**/tests/fourslash/**

1
server/.eslintignore Normal file
View File

@ -0,0 +1 @@
**/tests/fourslash/**

37
server/.vscode/launch.json vendored
View File

@ -16,6 +16,43 @@
"internalConsoleOptions": "neverOpen",
"disableOptimisticBPs": true,
"program": "${workspaceFolder}/node_modules/jest/bin/jest"
},
{
"type": "node",
"name": "jest current file",
"request": "launch",
"args": [
"${fileBasenameNoExtension}",
"--config",
"jest.config.js"
],
"cwd": "${workspaceFolder}",
"console": "integratedTerminal",
"internalConsoleOptions": "neverOpen",
"disableOptimisticBPs": true,
"program": "${workspaceFolder}/node_modules/jest/bin/jest",
"windows": {
"program": "${workspaceFolder}/node_modules/jest/bin/jest",
}
}
{
"type": "node",
"name": "fourslash current file",
"request": "launch",
"args": [
"fourslashrunner.test.ts",
"-t ${fileBasenameNoExtension}",
"--config",
"jest.config.js"
],
"cwd": "${workspaceFolder}",
"console": "integratedTerminal",
"internalConsoleOptions": "neverOpen",
"disableOptimisticBPs": true,
"program": "${workspaceFolder}/node_modules/jest/bin/jest",
"windows": {
"program": "${workspaceFolder}/node_modules/jest/bin/jest",
}
}
]
}

4
server/.vscode/settings.json vendored Normal file
View File

@ -0,0 +1,4 @@
{
"javascript.format.insertSpaceAfterOpeningAndBeforeClosingTemplateStringBraces": true,
"typescript.format.insertSpaceAfterOpeningAndBeforeClosingTemplateStringBraces": true
}

3
server/src/analyzer/binder.ts
View File

@ -19,7 +19,8 @@
import * as assert from 'assert';
import { DiagnosticLevel } from '../common/configOptions';
import { CreateTypeStubFileAction, getEmptyRange } from '../common/diagnostic';
import { CreateTypeStubFileAction } from '../common/diagnostic';
import { getEmptyRange } from '../common/textRange';
import { DiagnosticRule } from '../common/diagnosticRules';
import { convertOffsetsToRange } from '../common/positionUtils';
import { PythonVersion } from '../common/pythonVersion';

4
server/src/analyzer/declaration.ts
View File

@ -9,7 +9,7 @@
* is explicitly declared).
*/
import { DiagnosticTextRange } from '../common/diagnostic';
import { Range } from '../common/textRange';
import { ClassNode, ExpressionNode, FunctionNode, ImportAsNode,
ImportFromAsNode, ImportFromNode, ModuleNode, NameNode, ParameterNode,
ParseNode, ReturnNode, StringListNode, TypeAnnotationNode, YieldFromNode,
@ -38,7 +38,7 @@ export interface DeclarationBase {
// The file and range within that file that
// contains the declaration.
path: string;
range: DiagnosticTextRange;
range: Range;
}
export interface IntrinsicDeclaration extends DeclarationBase {

2
server/src/analyzer/declarationUtils.ts
View File

@ -38,7 +38,7 @@ export function areDeclarationsSame(decl1: Declaration, decl2: Declaration): boo
}
if (decl1.range.start.line !== decl2.range.start.line ||
decl1.range.start.column !== decl2.range.start.column) {
decl1.range.start.character !== decl2.range.start.character) {
return false;
}

102
server/src/analyzer/importResolver.ts
View File

@ -8,17 +8,18 @@
* runtime rules of Python.
*/
import * as fs from 'fs';
import { ConfigOptions, ExecutionEnvironment } from '../common/configOptions';
import { combinePaths, ensureTrailingDirectorySeparator, getDirectoryPath,
import {
combinePaths, ensureTrailingDirectorySeparator, getDirectoryPath,
getFileExtension, getFileSystemEntries, getPathComponents, isDirectory,
isFile, stripFileExtension, stripTrailingDirectorySeparator } from '../common/pathUtils';
isFile, stripFileExtension, stripTrailingDirectorySeparator
} from '../common/pathUtils';
import { versionToString } from '../common/pythonVersion';
import * as StringUtils from '../common/stringUtils';
import { ImplicitImport, ImportResult, ImportType } from './importResult';
import * as PythonPathUtils from './pythonPathUtils';
import { isDunderName } from './symbolNameUtils';
import { VirtualFileSystem } from '../common/vfs';
export interface ImportedModuleDescriptor {
leadingDots: number;
@ -41,7 +42,10 @@ export class ImportResolver {
private _cachedTypeshedStdLibPath: string | undefined;
private _cachedTypeshedThirdPartyPath: string | undefined;
constructor(configOptions: ConfigOptions) {
readonly fileSystem: VirtualFileSystem;
constructor(fs: VirtualFileSystem, configOptions: ConfigOptions) {
this.fileSystem = fs;
this._configOptions = configOptions;
}
@ -53,7 +57,7 @@ export class ImportResolver {
// Resolves the import and returns the path if it exists, otherwise
// returns undefined.
resolveImport(sourceFilePath: string, execEnv: ExecutionEnvironment,
moduleDescriptor: ImportedModuleDescriptor): ImportResult {
moduleDescriptor: ImportedModuleDescriptor): ImportResult {
const importName = this._formatImportName(moduleDescriptor);
const importFailureInfo: string[] = [];
@ -109,7 +113,7 @@ export class ImportResolver {
}
if (localImport && (bestResultSoFar === undefined ||
localImport.resolvedPaths.length > bestResultSoFar.resolvedPaths.length)) {
localImport.resolvedPaths.length > bestResultSoFar.resolvedPaths.length)) {
bestResultSoFar = localImport;
}
}
@ -157,7 +161,7 @@ export class ImportResolver {
}
if (bestResultSoFar === undefined ||
thirdPartyImport.resolvedPaths.length > bestResultSoFar.resolvedPaths.length) {
thirdPartyImport.resolvedPaths.length > bestResultSoFar.resolvedPaths.length) {
bestResultSoFar = thirdPartyImport;
}
}
@ -190,7 +194,7 @@ export class ImportResolver {
}
getCompletionSuggestions(sourceFilePath: string, execEnv: ExecutionEnvironment,
moduleDescriptor: ImportedModuleDescriptor, similarityLimit: number): string[] {
moduleDescriptor: ImportedModuleDescriptor, similarityLimit: number): string[] {
const importFailureInfo: string[] = [];
const suggestions: string[] = [];
@ -316,7 +320,7 @@ export class ImportResolver {
}
private _lookUpResultsInCache(execEnv: ExecutionEnvironment, importName: string,
importedSymbols: string[] | undefined) {
importedSymbols: string[] | undefined) {
const cacheForExecEnv = this._cachedImportResults.get(execEnv.root);
if (!cacheForExecEnv) {
@ -332,7 +336,7 @@ export class ImportResolver {
}
private _addResultsToCache(execEnv: ExecutionEnvironment, importName: string,
importResult: ImportResult, importedSymbols: string[] | undefined) {
importResult: ImportResult, importedSymbols: string[] | undefined) {
let cacheForExecEnv = this._cachedImportResults.get(execEnv.root);
if (!cacheForExecEnv) {
@ -346,7 +350,7 @@ export class ImportResolver {
}
private _getModuleNameFromPath(containerPath: string, filePath: string,
stripTopContainerDir = false): string | undefined {
stripTopContainerDir = false): string | undefined {
containerPath = ensureTrailingDirectorySeparator(containerPath);
let filePathWithoutExtension = stripFileExtension(filePath);
@ -378,21 +382,21 @@ export class ImportResolver {
}
private _getPythonSearchPaths(execEnv: ExecutionEnvironment,
importFailureInfo: string[]) {
importFailureInfo: string[]) {
const cacheKey = execEnv.venv ? execEnv.venv : '<default>';
// Find the site packages for the configured virtual environment.
if (!this._cachedPythonSearchPaths.has(cacheKey)) {
this._cachedPythonSearchPaths.set(cacheKey, PythonPathUtils.findPythonSearchPaths(
this._configOptions, execEnv.venv, importFailureInfo) || []);
this.fileSystem, this._configOptions, execEnv.venv, importFailureInfo) || []);
}
return this._cachedPythonSearchPaths.get(cacheKey)!;
}
private _findTypeshedPath(execEnv: ExecutionEnvironment, moduleDescriptor: ImportedModuleDescriptor,
importName: string, isStdLib: boolean, importFailureInfo: string[]): ImportResult | undefined {
importName: string, isStdLib: boolean, importFailureInfo: string[]): ImportResult | undefined {
importFailureInfo.push(`Looking for typeshed ${ isStdLib ? 'stdlib' : 'third_party' } path`);
@ -409,7 +413,7 @@ export class ImportResolver {
const pythonVersionString = minorVersion > 0 ? versionToString(0x300 + minorVersion) :
minorVersion === 0 ? '3' : '2and3';
const testPath = combinePaths(typeshedPath, pythonVersionString);
if (fs.existsSync(testPath)) {
if (this.fileSystem.existsSync(testPath)) {
const importInfo = this._resolveAbsoluteImport(testPath, moduleDescriptor,
importName, importFailureInfo);
if (importInfo && importInfo.isImportFound) {
@ -430,8 +434,8 @@ export class ImportResolver {
}
private _getCompletionSuggestionsTypeshedPath(execEnv: ExecutionEnvironment,
moduleDescriptor: ImportedModuleDescriptor, isStdLib: boolean,
suggestions: string[], similarityLimit: number) {
moduleDescriptor: ImportedModuleDescriptor, isStdLib: boolean,
suggestions: string[], similarityLimit: number) {
const importFailureInfo: string[] = [];
const typeshedPath = this._getTypeshedPath(isStdLib, execEnv, importFailureInfo);
@ -447,7 +451,7 @@ export class ImportResolver {
const pythonVersionString = minorVersion > 0 ? versionToString(0x300 + minorVersion) :
minorVersion === 0 ? '3' : '2and3';
const testPath = combinePaths(typeshedPath, pythonVersionString);
if (fs.existsSync(testPath)) {
if (this.fileSystem.existsSync(testPath)) {
this._getCompletionSuggestionsAbsolute(testPath, moduleDescriptor,
suggestions, similarityLimit);
}
@ -461,7 +465,7 @@ export class ImportResolver {
}
private _getTypeshedPath(isStdLib: boolean, execEnv: ExecutionEnvironment,
importFailureInfo: string[]) {
importFailureInfo: string[]) {
// See if we have it cached.
if (isStdLib) {
@ -480,14 +484,14 @@ export class ImportResolver {
// python search paths, then in the typeshed-fallback directory.
if (this._configOptions.typeshedPath) {
const possibleTypeshedPath = this._configOptions.typeshedPath;
if (fs.existsSync(possibleTypeshedPath) && isDirectory(possibleTypeshedPath)) {
if (this.fileSystem.existsSync(possibleTypeshedPath) && isDirectory(this.fileSystem, possibleTypeshedPath)) {
typeshedPath = possibleTypeshedPath;
}
} else {
const pythonSearchPaths = this._getPythonSearchPaths(execEnv, importFailureInfo);
for (const searchPath of pythonSearchPaths) {
const possibleTypeshedPath = combinePaths(searchPath, 'typeshed');
if (fs.existsSync(possibleTypeshedPath) && isDirectory(possibleTypeshedPath)) {
if (this.fileSystem.existsSync(possibleTypeshedPath) && isDirectory(this.fileSystem, possibleTypeshedPath)) {
typeshedPath = possibleTypeshedPath;
break;
}
@ -496,12 +500,12 @@ export class ImportResolver {
// If typeshed directory wasn't found in other locations, use the fallback.
if (!typeshedPath) {
typeshedPath = PythonPathUtils.getTypeShedFallbackPath() || '';
typeshedPath = PythonPathUtils.getTypeShedFallbackPath(this.fileSystem.getModulePath()) || '';
}
typeshedPath = PythonPathUtils.getTypeshedSubdirectory(typeshedPath, isStdLib);
if (!fs.existsSync(typeshedPath) || !isDirectory(typeshedPath)) {
if (!this.fileSystem.existsSync(typeshedPath) || !isDirectory(this.fileSystem, typeshedPath)) {
return undefined;
}
@ -516,8 +520,8 @@ export class ImportResolver {
}
private _resolveRelativeImport(sourceFilePath: string,
moduleDescriptor: ImportedModuleDescriptor, importName: string,
importFailureInfo: string[]): ImportResult | undefined {
moduleDescriptor: ImportedModuleDescriptor, importName: string,
importFailureInfo: string[]): ImportResult | undefined {
importFailureInfo.push('Attempting to resolve relative import');
@ -542,8 +546,8 @@ export class ImportResolver {
}
private _getCompletionSuggestionsRelative(sourceFilePath: string,
moduleDescriptor: ImportedModuleDescriptor, suggestions: string[],
similarityLimit: number) {
moduleDescriptor: ImportedModuleDescriptor, suggestions: string[],
similarityLimit: number) {
// Determine which search path this file is part of.
let curDir = getDirectoryPath(sourceFilePath);
@ -562,8 +566,8 @@ export class ImportResolver {
// Follows import resolution algorithm defined in PEP-420:
// https://www.python.org/dev/peps/pep-0420/
private _resolveAbsoluteImport(rootPath: string, moduleDescriptor: ImportedModuleDescriptor,
importName: string, importFailureInfo: string[], allowPartial = false,
allowPydFile = false, allowStubsFolder = false): ImportResult | undefined {
importName: string, importFailureInfo: string[], allowPartial = false,
allowPydFile = false, allowStubsFolder = false): ImportResult | undefined {
importFailureInfo.push(`Attempting to resolve using root path '${ rootPath }'`);
@ -582,14 +586,14 @@ export class ImportResolver {
const pyiFilePath = pyFilePath + 'i';
const pydFilePath = pyFilePath + 'd';
if (fs.existsSync(pyiFilePath) && isFile(pyiFilePath)) {
if (this.fileSystem.existsSync(pyiFilePath) && isFile(this.fileSystem, pyiFilePath)) {
importFailureInfo.push(`Resolved import with file '${ pyiFilePath }'`);
resolvedPaths.push(pyiFilePath);
isStubFile = true;
} else if (fs.existsSync(pyFilePath) && isFile(pyFilePath)) {
} else if (this.fileSystem.existsSync(pyFilePath) && isFile(this.fileSystem, pyFilePath)) {
importFailureInfo.push(`Resolved import with file '${ pyFilePath }'`);
resolvedPaths.push(pyFilePath);
} else if (allowPydFile && fs.existsSync(pydFilePath) && isFile(pydFilePath)) {
} else if (allowPydFile && this.fileSystem.existsSync(pydFilePath) && isFile(this.fileSystem, pydFilePath)) {
importFailureInfo.push(`Resolved import with file '${ pydFilePath }'`);
resolvedPaths.push(pydFilePath);
isPydFile = true;
@ -612,14 +616,14 @@ export class ImportResolver {
// the string '-stubs' to its top-level directory name. We'll
// look there first.
const stubsDirPath = dirPath + '-stubs';
foundDirectory = fs.existsSync(stubsDirPath) && isDirectory(stubsDirPath);
foundDirectory = this.fileSystem.existsSync(stubsDirPath) && isDirectory(this.fileSystem, stubsDirPath);
if (foundDirectory) {
dirPath = stubsDirPath;
}
}
if (!foundDirectory) {
foundDirectory = fs.existsSync(dirPath) && isDirectory(dirPath);
foundDirectory = this.fileSystem.existsSync(dirPath) && isDirectory(this.fileSystem, dirPath);
}
if (foundDirectory) {
@ -635,14 +639,14 @@ export class ImportResolver {
const pyiFilePath = pyFilePath + 'i';
let foundInit = false;
if (fs.existsSync(pyiFilePath) && isFile(pyiFilePath)) {
if (this.fileSystem.existsSync(pyiFilePath) && isFile(this.fileSystem, pyiFilePath)) {
importFailureInfo.push(`Resolved import with file '${ pyiFilePath }'`);
resolvedPaths.push(pyiFilePath);
if (isLastPart) {
isStubFile = true;
}
foundInit = true;
} else if (fs.existsSync(pyFilePath) && isFile(pyFilePath)) {
} else if (this.fileSystem.existsSync(pyFilePath) && isFile(this.fileSystem, pyFilePath)) {
importFailureInfo.push(`Resolved import with file '${ pyFilePath }'`);
resolvedPaths.push(pyFilePath);
foundInit = true;
@ -661,16 +665,16 @@ export class ImportResolver {
const pyiFilePath = pyFilePath + 'i';
const pydFilePath = pyFilePath + 'd';
if (fs.existsSync(pyiFilePath) && isFile(pyiFilePath)) {
if (this.fileSystem.existsSync(pyiFilePath) && isFile(this.fileSystem, pyiFilePath)) {
importFailureInfo.push(`Resolved import with file '${ pyiFilePath }'`);
resolvedPaths.push(pyiFilePath);
if (isLastPart) {
isStubFile = true;
}
} else if (fs.existsSync(pyFilePath) && isFile(pyFilePath)) {
} else if (this.fileSystem.existsSync(pyFilePath) && isFile(this.fileSystem, pyFilePath)) {
importFailureInfo.push(`Resolved import with file '${ pyFilePath }'`);
resolvedPaths.push(pyFilePath);
} else if (allowPydFile && fs.existsSync(pydFilePath) && isFile(pydFilePath)) {
} else if (allowPydFile && this.fileSystem.existsSync(pydFilePath) && isFile(this.fileSystem, pydFilePath)) {
importFailureInfo.push(`Resolved import with file '${ pydFilePath }'`);
resolvedPaths.push(pydFilePath);
if (isLastPart) {
@ -718,8 +722,8 @@ export class ImportResolver {
}
private _getCompletionSuggestionsAbsolute(rootPath: string,
moduleDescriptor: ImportedModuleDescriptor, suggestions: string[],
similarityLimit: number) {
moduleDescriptor: ImportedModuleDescriptor, suggestions: string[],
similarityLimit: number) {
// Starting at the specified path, walk the file system to find the
// specified module.
@ -746,7 +750,7 @@ export class ImportResolver {
}
dirPath = combinePaths(dirPath, nameParts[i]);
if (!fs.existsSync(dirPath) || !isDirectory(dirPath)) {
if (!this.fileSystem.existsSync(dirPath) || !isDirectory(this.fileSystem, dirPath)) {
break;
}
}
@ -754,9 +758,9 @@ export class ImportResolver {
}
private _addFilteredSuggestions(dirPath: string, filter: string, suggestions: string[],
similarityLimit: number) {
similarityLimit: number) {
const entries = getFileSystemEntries(dirPath);
const entries = getFileSystemEntries(this.fileSystem, dirPath);
entries.files.forEach(file => {
const fileWithoutExtension = stripFileExtension(file);
@ -765,7 +769,7 @@ export class ImportResolver {
if (fileExtension === '.py' || fileExtension === '.pyi' || fileExtension === '.pyd') {
if (fileWithoutExtension !== '__init__') {
if (!filter || StringUtils.computeCompletionSimilarity(
filter, fileWithoutExtension) >= similarityLimit) {
filter, fileWithoutExtension) >= similarityLimit) {
this._addUniqueSuggestion(fileWithoutExtension, suggestions);
}
@ -826,7 +830,7 @@ export class ImportResolver {
const implicitImportMap = new Map<string, ImplicitImport>();
// Enumerate all of the files and directories in the path.
const entries = getFileSystemEntries(dirPath);
const entries = getFileSystemEntries(this.fileSystem, dirPath);
// Add implicit file-based modules.
for (const fileName of entries.files) {
@ -857,10 +861,10 @@ export class ImportResolver {
let isStubFile = false;
let path = '';
if (fs.existsSync(pyiFilePath) && isFile(pyiFilePath)) {
if (this.fileSystem.existsSync(pyiFilePath) && isFile(this.fileSystem, pyiFilePath)) {
isStubFile = true;
path = pyiFilePath;
} else if (fs.existsSync(pyFilePath) && isFile(pyFilePath)) {
} else if (this.fileSystem.existsSync(pyFilePath) && isFile(this.fileSystem, pyFilePath)) {
path = pyFilePath;
}

8
server/src/analyzer/importStatementUtils.ts
View File

@ -8,7 +8,7 @@
* import statements in a python source file.
*/
import { DiagnosticTextPosition } from '../common/diagnostic';
import { Position } from '../common/textRange';
import { TextEditAction } from '../common/editAction';
import { convertOffsetToPosition } from '../common/positionUtils';
import { TextRange } from '../common/textRange';
@ -111,7 +111,7 @@ export function getTextEditsForAutoImportInsertion(symbolName: string, importSta
// We need to emit a new 'from import' statement.
let newImportStatement = `from ${ moduleName } import ${ symbolName }`;
let insertionPosition: DiagnosticTextPosition;
let insertionPosition: Position;
if (importStatements.orderedImports.length > 0) {
let insertBefore = true;
let insertionImport = importStatements.orderedImports[0];
@ -180,12 +180,12 @@ export function getTextEditsForAutoImportInsertion(symbolName: string, importSta
insertBefore ? insertionImport.node.start : TextRange.getEnd(insertionImport.node),
parseResults.tokenizerOutput.lines);
} else {
insertionPosition = { line: 0, column: 0 };
insertionPosition = { line: 0, character: 0 };
}
} else {
// Insert at or near the top of the file. See if there's a doc string and
// copyright notice, etc. at the top. If so, move past those.
insertionPosition = { line: 0, column: 0 };
insertionPosition = { line: 0, character: 0 };
let addNewLineBefore = false;
for (const statement of parseResults.parseTree.statements) {

4
server/src/analyzer/parseTreeUtils.ts
View File

@ -9,7 +9,7 @@
import * as assert from 'assert';
import { DiagnosticTextPosition } from '../common/diagnostic';
import { Position } from '../common/textRange';
import { convertPositionToOffset } from '../common/positionUtils';
import { TextRange } from '../common/textRange';
import { TextRangeCollection } from '../common/textRangeCollection';
@ -40,7 +40,7 @@ export function getNodeDepth(node: ParseNode): number {
}
// Returns the deepest node that contains the specified position.
export function findNodeByPosition(node: ParseNode, position: DiagnosticTextPosition,
export function findNodeByPosition(node: ParseNode, position: Position,
lines: TextRangeCollection<TextRange>): ParseNode | undefined {
const offset = convertPositionToOffset(position, lines);

77
server/src/analyzer/program.ts
View File

@ -13,12 +13,13 @@ import { CompletionItem, CompletionList, DocumentSymbol, SymbolInformation } fro
import { ConfigOptions } from '../common/configOptions';
import { ConsoleInterface, StandardConsole } from '../common/console';
import { Diagnostic, DiagnosticTextPosition,
DiagnosticTextRange, DocumentTextRange, doRangesOverlap } from '../common/diagnostic';
import { Diagnostic } from '../common/diagnostic';
import { FileDiagnostics } from '../common/diagnosticSink';
import { FileEditAction, TextEditAction } from '../common/editAction';
import { combinePaths, getDirectoryPath, getRelativePath, makeDirectories,
normalizePath, stripFileExtension } from '../common/pathUtils';
import {
combinePaths, getDirectoryPath, getRelativePath, makeDirectories,
normalizePath, stripFileExtension
} from '../common/pathUtils';
import { Duration, timingStats } from '../common/timing';
import { ModuleSymbolMap } from '../languageService/completionProvider';
import { HoverResults } from '../languageService/hoverProvider';
@ -33,6 +34,7 @@ import { SourceFile } from './sourceFile';
import { SymbolTable } from './symbol';
import { createTypeEvaluator, TypeEvaluator } from './typeEvaluator';
import { TypeStubWriter } from './typeStubWriter';
import { Position, Range, DocumentRange, doRangesOverlap } from '../common/textRange';
const _maxImportDepth = 256;
@ -81,8 +83,7 @@ export class Program {
private _importResolver: ImportResolver;
constructor(initialImportResolver: ImportResolver, initialConfigOptions: ConfigOptions,
console?: ConsoleInterface) {
console?: ConsoleInterface) {
this._console = console || new StandardConsole();
this._evaluator = createTypeEvaluator(this._lookUpImport);
this._importResolver = initialImportResolver;
@ -140,8 +141,8 @@ export class Program {
this._sourceFileList.forEach(fileInfo => {
if (fileInfo.sourceFile.isParseRequired() ||
fileInfo.sourceFile.isBindingRequired() ||
fileInfo.sourceFile.isCheckingRequired()) {
fileInfo.sourceFile.isBindingRequired() ||
fileInfo.sourceFile.isCheckingRequired()) {
if ((!this._configOptions.checkOnlyOpenFiles && fileInfo.isTracked) || fileInfo.isOpenByClient) {
sourceFileCount++;
@ -169,7 +170,7 @@ export class Program {
return sourceFileInfo.sourceFile;
}
const sourceFile = new SourceFile(filePath, false, false, this._console);
const sourceFile = new SourceFile(this._fs, filePath, false, false, this._console);
sourceFileInfo = {
sourceFile,
isTracked: true,
@ -187,7 +188,7 @@ export class Program {
setFileOpened(filePath: string, version: number | null, contents: string) {
let sourceFileInfo = this._sourceFileMap.get(filePath);
if (!sourceFileInfo) {
const sourceFile = new SourceFile(filePath, false, false, this._console);
const sourceFile = new SourceFile(this._fs, filePath, false, false, this._console);
sourceFileInfo = {
sourceFile,
isTracked: false,
@ -388,20 +389,23 @@ export class Program {
const typeStubDir = getDirectoryPath(typeStubPath);
try {
makeDirectories(typeStubDir, typingsPath);
makeDirectories(this._fs, typeStubDir, typingsPath);
} catch (e) {
const errMsg = `Could not create directory for '${ typeStubDir }'`;
throw new Error(errMsg);
}
this._bindFile(sourceFileInfo);
const writer = new TypeStubWriter(typeStubPath,
sourceFileInfo.sourceFile, this._evaluator);
const writer = new TypeStubWriter(typeStubPath, sourceFileInfo.sourceFile, this._evaluator);
writer.write();
}
}
}
private get _fs() {
return this._importResolver.fileSystem;
}
private _createNewEvaluator() {
this._evaluator = createTypeEvaluator(this._lookUpImport);
}
@ -550,7 +554,7 @@ export class Program {
// have already been checked (they and their recursive imports have completed the
// check phase), they are not included in the results.
private _getImportsRecursive(file: SourceFileInfo, closureMap: Map<string, SourceFileInfo>,
recursionCount: number) {
recursionCount: number) {
// If the file is already in the closure map, we found a cyclical
// dependency. Don't recur further.
@ -576,8 +580,8 @@ export class Program {
}
private _detectAndReportImportCycles(sourceFileInfo: SourceFileInfo,
dependencyChain: SourceFileInfo[] = [],
dependencyMap = new Map<string, boolean>()): void {
dependencyChain: SourceFileInfo[] = [],
dependencyMap = new Map<string, boolean>()): void {
// Don't bother checking for typestub files or third-party files.
if (sourceFileInfo.sourceFile.isStubFile() || sourceFileInfo.isThirdPartyImport) {
@ -631,7 +635,7 @@ export class Program {
}
private _markFileDirtyRecursive(sourceFileInfo: SourceFileInfo,
markMap: Map<string, boolean>) {
markMap: Map<string, boolean>) {
const filePath = sourceFileInfo.sourceFile.getFilePath();
@ -652,7 +656,7 @@ export class Program {
this._sourceFileList.forEach(sourceFileInfo => {
if ((!options.checkOnlyOpenFiles && sourceFileInfo.isTracked) || sourceFileInfo.isOpenByClient) {
const diagnostics = sourceFileInfo.sourceFile.getDiagnostics(
options, sourceFileInfo.diagnosticsVersion);
options, sourceFileInfo.diagnosticsVersion);
if (diagnostics !== undefined) {
fileDiagnostics.push({
filePath: sourceFileInfo.sourceFile.getFilePath(),
@ -670,7 +674,7 @@ export class Program {
return fileDiagnostics;
}
getDiagnosticsForRange(filePath: string, options: ConfigOptions, range: DiagnosticTextRange): Diagnostic[] {
getDiagnosticsForRange(filePath: string, options: ConfigOptions, range: Range): Diagnostic[] {
const sourceFile = this.getSourceFile(filePath);
if (!sourceFile) {
return [];
@ -686,8 +690,8 @@ export class Program {
});
}
getDefinitionsForPosition(filePath: string, position: DiagnosticTextPosition):
DocumentTextRange[] | undefined {
getDefinitionsForPosition(filePath: string, position: Position):
DocumentRange[] | undefined {
const sourceFileInfo = this._sourceFileMap.get(filePath);
if (!sourceFileInfo) {
@ -699,8 +703,8 @@ export class Program {
return sourceFileInfo.sourceFile.getDefinitionsForPosition(position, this._evaluator);
}
getReferencesForPosition(filePath: string, position: DiagnosticTextPosition,
includeDeclaration: boolean): DocumentTextRange[] | undefined {
getReferencesForPosition(filePath: string, position: Position,
includeDeclaration: boolean): DocumentRange[] | undefined {
const sourceFileInfo = this._sourceFileMap.get(filePath);
if (!sourceFileInfo) {
@ -756,8 +760,8 @@ export class Program {
}
}
getHoverForPosition(filePath: string, position: DiagnosticTextPosition):
HoverResults | undefined {
getHoverForPosition(filePath: string, position: Position):
HoverResults | undefined {
const sourceFileInfo = this._sourceFileMap.get(filePath);
if (!sourceFileInfo) {
@ -769,8 +773,8 @@ export class Program {
return sourceFileInfo.sourceFile.getHoverForPosition(position, this._evaluator);
}
getSignatureHelpForPosition(filePath: string, position: DiagnosticTextPosition):
SignatureHelpResults | undefined {
getSignatureHelpForPosition(filePath: string, position: Position):
SignatureHelpResults | undefined {
const sourceFileInfo = this._sourceFileMap.get(filePath);
if (!sourceFileInfo) {
@ -783,8 +787,8 @@ export class Program {
position, this._lookUpImport, this._evaluator);
}
getCompletionsForPosition(filePath: string, position: DiagnosticTextPosition,
workspacePath: string): CompletionList | undefined {
getCompletionsForPosition(filePath: string, position: Position,
workspacePath: string): CompletionList | undefined {
const sourceFileInfo = this._sourceFileMap.get(filePath);
if (!sourceFileInfo) {
@ -813,7 +817,7 @@ export class Program {
}
performQuickAction(filePath: string, command: string,
args: any[]): TextEditAction[] | undefined {
args: any[]): TextEditAction[] | undefined {
const sourceFileInfo = this._sourceFileMap.get(filePath);
if (!sourceFileInfo) {
@ -826,8 +830,8 @@ export class Program {
command, args);
}
renameSymbolAtPosition(filePath: string, position: DiagnosticTextPosition,
newName: string): FileEditAction[] | undefined {
renameSymbolAtPosition(filePath: string, position: Position,
newName: string): FileEditAction[] | undefined {
const sourceFileInfo = this._sourceFileMap.get(filePath);
if (!sourceFileInfo) {
@ -874,7 +878,7 @@ export class Program {
// If a file is no longer tracked or opened, it can
// be removed from the program.
for (let i = 0; i < this._sourceFileList.length; ) {
for (let i = 0; i < this._sourceFileList.length;) {
const fileInfo = this._sourceFileList[i];
if (!this._isFileNeeded(fileInfo)) {
fileDiagnostics.push({
@ -972,12 +976,12 @@ export class Program {
}
private _isImportAllowed(importer: SourceFileInfo, importResult: ImportResult,
isImportStubFile: boolean): boolean {
isImportStubFile: boolean): boolean {
let thirdPartyImportAllowed = this._configOptions.useLibraryCodeForTypes;
if (importResult.importType === ImportType.ThirdParty ||
(importer.isThirdPartyImport && importResult.importType === ImportType.Local)) {
(importer.isThirdPartyImport && importResult.importType === ImportType.Local)) {
if (this._allowedThirdPartyImports) {
if (importResult.isRelative) {
@ -1015,7 +1019,7 @@ export class Program {
}
private _updateSourceFileImports(sourceFileInfo: SourceFileInfo,
options: ConfigOptions): SourceFileInfo[] {
options: ConfigOptions): SourceFileInfo[] {
const filesAdded: SourceFileInfo[] = [];
@ -1086,6 +1090,7 @@ export class Program {
importedFileInfo = this._sourceFileMap.get(importPath)!;
} else {
const sourceFile = new SourceFile(
this._fs,
importPath, importInfo.isTypeshedFile,
importInfo.isThirdPartyImport, this._console);
importedFileInfo = {

34
server/src/analyzer/pythonPathUtils.ts
View File

@ -8,19 +8,16 @@
*/
import * as child_process from 'child_process';
import * as fs from 'fs';
import { ConfigOptions } from '../common/configOptions';
import { combinePaths, ensureTrailingDirectorySeparator, getDirectoryPath,
getFileSystemEntries, isDirectory, normalizePath } from '../common/pathUtils';
import {
combinePaths, ensureTrailingDirectorySeparator, getDirectoryPath,
getFileSystemEntries, isDirectory, normalizePath
} from '../common/pathUtils';
import { VirtualFileSystem } from '../common/vfs';
const cachedSearchPaths = new Map<string, string[]>();
export function getTypeShedFallbackPath() {
// The entry point to the tool should have set the __rootDirectory
// global variable to point to the directory that contains the
// typeshed-fallback directory.
let moduleDirectory = (global as any).__rootDirectory;
export function getTypeShedFallbackPath(moduleDirectory?: string) {
if (moduleDirectory) {
moduleDirectory = normalizePath(moduleDirectory);
return combinePaths(getDirectoryPath(
@ -35,8 +32,8 @@ export function getTypeshedSubdirectory(typeshedPath: string, isStdLib: boolean)
return combinePaths(typeshedPath, isStdLib ? 'stdlib' : 'third_party');
}
export function findPythonSearchPaths(configOptions: ConfigOptions, venv: string | undefined,
importFailureInfo: string[]): string[] | undefined {
export function findPythonSearchPaths(fs: VirtualFileSystem, configOptions: ConfigOptions,
venv: string | undefined, importFailureInfo: string[]): string[] | undefined {
importFailureInfo.push('Finding python search paths');
@ -77,7 +74,7 @@ export function findPythonSearchPaths(configOptions: ConfigOptions, venv: string
// We didn't find a site-packages directory directly in the lib
// directory. Scan for a "python*" directory instead.
const entries = getFileSystemEntries(libPath);
const entries = getFileSystemEntries(this._fs, libPath);
for (let i = 0; i < entries.directories.length; i++) {
const dirName = entries.directories[i];
if (dirName.startsWith('python')) {
@ -96,11 +93,12 @@ export function findPythonSearchPaths(configOptions: ConfigOptions, venv: string
}
// Fall back on the python interpreter.
return getPythonPathFromPythonInterpreter(configOptions.pythonPath, importFailureInfo);
return getPythonPathFromPythonInterpreter(fs, configOptions.pythonPath, importFailureInfo);
}
export function getPythonPathFromPythonInterpreter(interpreterPath: string | undefined,
importFailureInfo: string[]): string[] {
export function getPythonPathFromPythonInterpreter(fs: VirtualFileSystem,
interpreterPath: string | undefined,
importFailureInfo: string[]): string[] {
const searchKey = interpreterPath || '';
@ -116,9 +114,9 @@ export function getPythonPathFromPythonInterpreter(interpreterPath: string | und
// Set the working directory to a known location within
// the extension directory. Otherwise the execution of
// python can have unintended and surprising results.
const moduleDirectory = (global as any).__rootDirectory;
const moduleDirectory = fs.getModulePath();
if (moduleDirectory) {
process.chdir(moduleDirectory);
fs.chdir(moduleDirectory);
}
const commandLineArgs: string[] = ['-c', 'import sys, json; json.dump(sys.path, sys.stdout)'];
@ -143,7 +141,7 @@ export function getPythonPathFromPythonInterpreter(interpreterPath: string | und
const normalizedPath = normalizePath(execSplitEntry);
// Make sure the path exists and is a directory. We don't currently
// support zip files and other formats.
if (fs.existsSync(normalizedPath) && isDirectory(normalizedPath)) {
if (fs.existsSync(normalizedPath) && isDirectory(fs, normalizedPath)) {
pythonPaths.push(normalizedPath);
} else {
importFailureInfo.push(`Skipping '${ normalizedPath }' because it is not a valid directory`);

159
server/src/analyzer/service.ts
View File

@ -7,31 +7,28 @@
* A persistent service that is able to analyze a collection of
* Python files.
*/
import * as assert from 'assert';
import * as chokidar from 'chokidar';
import * as fs from 'fs';
import { CompletionItem, CompletionList, DocumentSymbol, SymbolInformation } from 'vscode-languageserver';
import { CommandLineOptions } from '../common/commandLineOptions';
import { ConfigOptions } from '../common/configOptions';
import { ConsoleInterface, StandardConsole } from '../common/console';
import { Diagnostic, DiagnosticTextPosition, DiagnosticTextRange,
DocumentTextRange } from '../common/diagnostic';
import { Diagnostic } from '../common/diagnostic';
import { FileDiagnostics } from '../common/diagnosticSink';
import { FileEditAction, TextEditAction } from '../common/editAction';
import { combinePaths, FileSpec, forEachAncestorDirectory, getDirectoryPath,
import {
combinePaths, FileSpec, forEachAncestorDirectory, getDirectoryPath,
getFileName, getFileSpec, getFileSystemEntries, isDirectory,
normalizePath, stripFileExtension } from '../common/pathUtils';
normalizePath, stripFileExtension
} from '../common/pathUtils';
import { Duration, timingStats } from '../common/timing';
import { HoverResults } from '../languageService/hoverProvider';
import { SignatureHelpResults } from '../languageService/signatureHelpProvider';
import { ImportedModuleDescriptor, ImportResolver } from './importResolver';
import { MaxAnalysisTime, Program } from './program';
import * as PythonPathUtils from './pythonPathUtils';
const _isMacintosh = process.platform === 'darwin';
const _isLinux = process.platform === 'linux';
import { Position, Range, DocumentRange } from '../common/textRange';
import { VirtualFileSystem, FileWatcher } from '../common/vfs';
export { MaxAnalysisTime } from './program';
@ -59,10 +56,10 @@ export class AnalyzerService {
private _typeStubTargetPath: string | undefined;
private _typeStubTargetIsSingleFile = false;
private _console: ConsoleInterface;
private _sourceFileWatcher: fs.FSWatcher | undefined;
private _sourceFileWatcher: FileWatcher | undefined;
private _reloadConfigTimer: any;
private _configFilePath: string | undefined;
private _configFileWatcher: fs.FSWatcher | undefined;
private _configFileWatcher: FileWatcher | undefined;
private _onCompletionCallback: AnalysisCompleteCallback | undefined;
private _watchForSourceChanges = false;
private _verboseOutput = false;
@ -71,11 +68,11 @@ export class AnalyzerService {
private _requireTrackedFileUpdate = true;
private _lastUserInteractionTime = Date.now();
constructor(instanceName: string, console?: ConsoleInterface) {
constructor(instanceName: string, fs: VirtualFileSystem, console?: ConsoleInterface, configOptions?: ConfigOptions) {
this._instanceName = instanceName;
this._console = console || new StandardConsole();
this._configOptions = new ConfigOptions(process.cwd());
this._importResolver = new ImportResolver(this._configOptions);
this._configOptions = configOptions ?? new ConfigOptions(process.cwd());
this._importResolver = new ImportResolver(fs, this._configOptions);
this._program = new Program(this._importResolver, this._configOptions, this._console);
this._executionRootPath = '';
this._typeStubTargetImportName = undefined;
@ -104,7 +101,7 @@ export class AnalyzerService {
this._typeStubTargetImportName = commandLineOptions.typeStubTargetImportName;
this._executionRootPath = normalizePath(combinePaths(
commandLineOptions.executionRoot, this._configOptions.projectRoot));
commandLineOptions.executionRoot, this._configOptions.projectRoot));
this._applyConfigOptions();
}
@ -125,14 +122,14 @@ export class AnalyzerService {
this._scheduleReanalysis(false);
}
getDefinitionForPosition(filePath: string, position: DiagnosticTextPosition):
DocumentTextRange[] | undefined {
getDefinitionForPosition(filePath: string, position: Position):
DocumentRange[] | undefined {
return this._program.getDefinitionsForPosition(filePath, position);
}
getReferencesForPosition(filePath: string, position: DiagnosticTextPosition,
includeDeclaration: boolean): DocumentTextRange[] | undefined {
getReferencesForPosition(filePath: string, position: Position,
includeDeclaration: boolean): DocumentRange[] | undefined {
return this._program.getReferencesForPosition(filePath, position, includeDeclaration);
}
@ -145,20 +142,20 @@ export class AnalyzerService {
this._program.addSymbolsForWorkspace(symbolList, query);
}
getHoverForPosition(filePath: string, position: DiagnosticTextPosition):
HoverResults | undefined {
getHoverForPosition(filePath: string, position: Position):
HoverResults | undefined {
return this._program.getHoverForPosition(filePath, position);
}
getSignatureHelpForPosition(filePath: string, position: DiagnosticTextPosition):
SignatureHelpResults | undefined {
getSignatureHelpForPosition(filePath: string, position: Position):
SignatureHelpResults | undefined {
return this._program.getSignatureHelpForPosition(filePath, position);
}
getCompletionsForPosition(filePath: string, position: DiagnosticTextPosition,
workspacePath: string): CompletionList | undefined {
getCompletionsForPosition(filePath: string, position: Position,
workspacePath: string): CompletionList | undefined {
return this._program.getCompletionsForPosition(filePath, position, workspacePath);
}
@ -171,8 +168,8 @@ export class AnalyzerService {
return this._program.performQuickAction(filePath, command, args);
}
renameSymbolAtPosition(filePath: string, position: DiagnosticTextPosition,
newName: string): FileEditAction[] | undefined {
renameSymbolAtPosition(filePath: string, position: Position,
newName: string): FileEditAction[] | undefined {
return this._program.renameSymbolAtPosition(filePath, position, newName);
}
@ -197,7 +194,7 @@ export class AnalyzerService {
return this._getFileNamesFromFileSpecs();
}
getDiagnosticsForRange(filePath: string, range: DiagnosticTextRange): Diagnostic[] {
getDiagnosticsForRange(filePath: string, range: Range): Diagnostic[] {
return this._program.getDiagnosticsForRange(filePath, this._configOptions, range);
}
@ -230,7 +227,7 @@ export class AnalyzerService {
// or a file.
configFilePath = combinePaths(commandLineOptions.executionRoot,
normalizePath(commandLineOptions.configFilePath));
if (!fs.existsSync(configFilePath)) {
if (!this._fs.existsSync(configFilePath)) {
this._console.log(`Configuration file not found at ${ configFilePath }.`);
configFilePath = commandLineOptions.executionRoot;
} else {
@ -350,7 +347,7 @@ export class AnalyzerService {
// Do some sanity checks on the specified settings and report missing
// or inconsistent information.
if (configOptions.venvPath) {
if (!fs.existsSync(configOptions.venvPath) || !isDirectory(configOptions.venvPath)) {
if (!this._fs.existsSync(configOptions.venvPath) || !isDirectory(this._fs, configOptions.venvPath)) {
this._console.log(
`venvPath ${ configOptions.venvPath } is not a valid directory.`);
}
@ -358,15 +355,13 @@ export class AnalyzerService {
if (configOptions.defaultVenv) {
const fullVenvPath = combinePaths(configOptions.venvPath, configOptions.defaultVenv);
if (!fs.existsSync(fullVenvPath) || !isDirectory(fullVenvPath)) {
if (!this._fs.existsSync(fullVenvPath) || !isDirectory(this._fs, fullVenvPath)) {
this._console.log(
`venv ${ configOptions.defaultVenv } subdirectory not found ` +
`in venv path ${ configOptions.venvPath }.`);
} else {
const importFailureInfo: string[] = [];
if (PythonPathUtils.findPythonSearchPaths(configOptions, undefined,
importFailureInfo) === undefined) {
if (PythonPathUtils.findPythonSearchPaths(this._fs, configOptions, undefined, importFailureInfo) === undefined) {
this._console.log(
`site-packages directory cannot be located for venvPath ` +
`${ configOptions.venvPath } and venv ${ configOptions.defaultVenv }.`);
@ -381,8 +376,7 @@ export class AnalyzerService {
}
} else {
const importFailureInfo: string[] = [];
const pythonPaths = PythonPathUtils.getPythonPathFromPythonInterpreter(
configOptions.pythonPath, importFailureInfo);
const pythonPaths = PythonPathUtils.getPythonPathFromPythonInterpreter(this._fs, configOptions.pythonPath, importFailureInfo);
if (pythonPaths.length === 0) {
if (configOptions.verboseOutput) {
this._console.log(
@ -418,14 +412,14 @@ export class AnalyzerService {
}
if (configOptions.typeshedPath) {
if (!fs.existsSync(configOptions.typeshedPath) || !isDirectory(configOptions.typeshedPath)) {
if (!this._fs.existsSync(configOptions.typeshedPath) || !isDirectory(this._fs, configOptions.typeshedPath)) {
this._console.log(
`typeshedPath ${ configOptions.typeshedPath } is not a valid directory.`);
}
}
if (configOptions.typingsPath) {
if (!fs.existsSync(configOptions.typingsPath) || !isDirectory(configOptions.typingsPath)) {
if (!this._fs.existsSync(configOptions.typingsPath) || !isDirectory(this._fs, configOptions.typingsPath)) {
this._console.log(
`typingsPath ${ configOptions.typingsPath } is not a valid directory.`);
}
@ -463,8 +457,8 @@ export class AnalyzerService {
try {
// Generate a new typings directory if necessary.
if (!fs.existsSync(typingsPath)) {
fs.mkdirSync(typingsPath);
if (!this._fs.existsSync(typingsPath)) {
this._fs.mkdirSync(typingsPath);
}
} catch (e) {
const errMsg = `Could not create typings directory '${ typingsPath }'`;
@ -476,8 +470,8 @@ export class AnalyzerService {
const typingsSubdirPath = combinePaths(typingsPath, typeStubInputTargetParts[0]);
try {
// Generate a new typings subdirectory if necessary.
if (!fs.existsSync(typingsSubdirPath)) {
fs.mkdirSync(typingsSubdirPath);
if (!this._fs.existsSync(typingsSubdirPath)) {
this._fs.mkdirSync(typingsSubdirPath);
}
} catch (e) {
const errMsg = `Could not create typings subdirectory '${ typingsSubdirPath }'`;
@ -500,6 +494,10 @@ export class AnalyzerService {
this._program.markAllFilesDirty(true);
}
private get _fs() {
return this._importResolver.fileSystem;
}
private _findConfigFileHereOrUp(searchPath: string): string | undefined {
return forEachAncestorDirectory(searchPath, ancestor => this._findConfigFile(ancestor));
}
@ -507,8 +505,9 @@ export class AnalyzerService {
private _findConfigFile(searchPath: string): string | undefined {
for (const name of _configFileNames) {
const fileName = combinePaths(searchPath, name);
if (fs.existsSync(fileName))
if (this._fs.existsSync(fileName)) {
return fileName;
}
}
return undefined;
}
@ -520,7 +519,7 @@ export class AnalyzerService {
while (true) {
// Attempt to read the config file contents.
try {
configContents = fs.readFileSync(configPath, { encoding: 'utf8' });
configContents = this._fs.readFileSync(configPath, 'utf8');
} catch {
this._console.log(`Config file "${ configPath }" could not be read.`);
this._reportConfigParseError();
@ -562,7 +561,7 @@ export class AnalyzerService {
this._configOptions.exclude);
for (const file of matchedFiles) {
fileMap.set(file, file);
fileMap.set(file, file);
}
});
@ -594,7 +593,7 @@ export class AnalyzerService {
// Namespace packages resolve to a directory name, so
// don't include those.
const resolvedPath = importResult.resolvedPaths[
importResult.resolvedPaths.length - 1];
importResult.resolvedPaths.length - 1];
// Get the directory that contains the root package.
let targetPath = getDirectoryPath(resolvedPath);
@ -610,7 +609,7 @@ export class AnalyzerService {
}
}
if (isDirectory(targetPath)) {
if (isDirectory(this._fs, targetPath)) {
this._typeStubTargetPath = targetPath;
}
@ -661,7 +660,7 @@ export class AnalyzerService {
const results: string[] = [];
const visitDirectory = (absolutePath: string, includeRegExp: RegExp) => {
const { files, directories } = getFileSystemEntries(absolutePath);
const { files, directories } = getFileSystemEntries(this._fs, absolutePath);
for (const file of files) {
const filePath = combinePaths(absolutePath, file);
@ -688,7 +687,7 @@ export class AnalyzerService {
if (!this._isInExcludePath(includeSpec.wildcardRoot, exclude)) {
try {
const stat = fs.statSync(includeSpec.wildcardRoot);
const stat = this._fs.statSync(includeSpec.wildcardRoot);
if (stat.isFile()) {
if (includeFileRegex.test(includeSpec.wildcardRoot)) {
results.push(includeSpec.wildcardRoot);
@ -740,7 +739,7 @@ export class AnalyzerService {
this._console.log(`Adding fs watcher for directories:\n ${ fileList.join('\n') }`);
}
this._sourceFileWatcher = this._createFileSystemWatcher(fileList).on('all', (event, path) => {
this._sourceFileWatcher = this._fs.createFileSystemWatcher(fileList, 'all', (event, path) => {
if (this._verboseOutput) {
this._console.log(`Received fs event '${ event }' for path '${ path }'`);
}
@ -765,59 +764,17 @@ export class AnalyzerService {
}
}
private _createFileSystemWatcher(paths: string[]): chokidar.FSWatcher {
// The following options are copied from VS Code source base. It also
// uses chokidar for its file watching.
const watcherOptions: chokidar.WatchOptions = {
ignoreInitial: true,
ignorePermissionErrors: true,
followSymlinks: true, // this is the default of chokidar and supports file events through symlinks
interval: 1000, // while not used in normal cases, if any error causes chokidar to fallback to polling, increase its intervals
binaryInterval: 1000,
disableGlobbing: true // fix https://github.com/Microsoft/vscode/issues/4586
};
if (_isMacintosh) {
// Explicitly disable on MacOS because it uses up large amounts of memory
// and CPU for large file hierarchies, resulting in instability and crashes.
watcherOptions.usePolling = false;
}
const excludes: string[] = [];
if (_isMacintosh || _isLinux) {
if (paths.some(path => path === '' || path === '/')) {
excludes.push('/dev/**');
if (_isLinux) {
excludes.push('/proc/**', '/sys/**');
}
}
}
watcherOptions.ignored = excludes;
const watcher = chokidar.watch(paths, watcherOptions);
watcher.on('error', _ => {
this._console.log('Error returned from file system watcher.');
});
// Detect if for some reason the native watcher library fails to load
if (_isMacintosh && !watcher.options.useFsEvents) {
this._console.log('Watcher could not use native fsevents library. File system watcher disabled.');
}
return watcher;
}
private _updateConfigFileWatcher() {
this._removeConfigFileWatcher();
if (this._configFilePath) {
this._configFileWatcher = this._createFileSystemWatcher([this._configFilePath])
.on('all', event => {
if (this._verboseOutput) {
this._console.log(`Received fs event '${ event }' for config file`);
}
this._scheduleReloadConfigFile();
});
this._configFileWatcher = this._fs.createFileSystemWatcher([this._configFilePath],
'all', event => {
if (this._verboseOutput) {
this._console.log(`Received fs event '${ event }' for config file`);
}
this._scheduleReloadConfigFile();
});
}
}
@ -858,7 +815,7 @@ export class AnalyzerService {
private _applyConfigOptions() {
// Allocate a new import resolver because the old one has information
// cached based on the previous config options.
this._importResolver = new ImportResolver(this._configOptions);
this._importResolver = new ImportResolver(this._fs, this._configOptions);
this._program.setImportResolver(this._importResolver);
this._updateSourceFileWatchers();

86
server/src/analyzer/sourceFile.ts
View File

@ -8,19 +8,19 @@
*/
import * as assert from 'assert';
import * as fs from 'fs';
import { CompletionItem, CompletionList, DocumentSymbol, SymbolInformation } from 'vscode-languageserver';
import { ConfigOptions, ExecutionEnvironment,
getDefaultDiagnosticSettings } from '../common/configOptions';
import {
ConfigOptions, ExecutionEnvironment,
getDefaultDiagnosticSettings
} from '../common/configOptions';
import { ConsoleInterface, StandardConsole } from '../common/console';
import { Diagnostic, DiagnosticCategory, DiagnosticTextPosition,
DocumentTextRange, getEmptyRange } from '../common/diagnostic';
import { Diagnostic, DiagnosticCategory } from '../common/diagnostic';
import { DiagnosticSink, TextRangeDiagnosticSink } from '../common/diagnosticSink';
import { TextEditAction } from '../common/editAction';
import { getFileName, normalizeSlashes } from '../common/pathUtils';
import * as StringUtils from '../common/stringUtils';
import { TextRange } from '../common/textRange';
import { TextRange, getEmptyRange, Position, DocumentRange } from '../common/textRange';
import { TextRangeCollection } from '../common/textRangeCollection';
import { timingStats } from '../common/timing';
import { CompletionItemData, CompletionProvider, ModuleSymbolMap } from '../languageService/completionProvider';
@ -46,6 +46,7 @@ import { Scope } from './scope';
import { SymbolTable } from './symbol';
import { TestWalker } from './testWalker';
import { TypeEvaluator } from './typeEvaluator';
import { VirtualFileSystem } from '../common/vfs';
const _maxImportCyclesPerFile = 4;
@ -139,9 +140,12 @@ export class SourceFile {
private _typingModulePath?: string;
private _collectionsModulePath?: string;
constructor(filePath: string, isTypeshedStubFile: boolean, isThirdPartyImport: boolean,
console?: ConsoleInterface) {
readonly fileSystem: VirtualFileSystem;
constructor(fs: VirtualFileSystem, filePath: string, isTypeshedStubFile: boolean,
isThirdPartyImport: boolean, console?: ConsoleInterface) {
this.fileSystem = fs;
this._console = console || new StandardConsole();
this._filePath = filePath;
this._isStubFile = filePath.endsWith('.pyi');
@ -154,11 +158,11 @@ export class SourceFile {
this._isBuiltInStubFile = false;
if (this._isStubFile) {
if (this._filePath.endsWith(normalizeSlashes('/collections/__init__.pyi')) ||
fileName === 'builtins.pyi' ||
fileName === '_importlib_modulespec.pyi' ||
fileName === 'dataclasses.pyi' ||
fileName === 'abc.pyi' ||
fileName === 'enum.pyi') {
fileName === 'builtins.pyi' ||
fileName === '_importlib_modulespec.pyi' ||
fileName === 'dataclasses.pyi' ||
fileName === 'abc.pyi' ||
fileName === 'enum.pyi') {
this._isBuiltInStubFile = true;
}
@ -181,7 +185,7 @@ export class SourceFile {
// If the prevVersion is specified, the method returns undefined if
// the diagnostics haven't changed.
getDiagnostics(options: ConfigOptions, prevDiagnosticVersion?: number):
Diagnostic[] | undefined {
Diagnostic[] | undefined {
if (this._diagnosticVersion === prevDiagnosticVersion) {
return undefined;
@ -302,7 +306,7 @@ export class SourceFile {
// that of the previous contents.
try {
// Read the file's contents.
const fileContents = fs.readFileSync(this._filePath, { encoding: 'utf8' });
const fileContents = this.fileSystem.readFileSync(this._filePath, 'utf8');
if (fileContents.length !== this._lastFileContentLength) {
return true;
@ -432,7 +436,7 @@ export class SourceFile {
try {
timingStats.readFileTime.timeOperation(() => {
// Read the file's contents.
fileContents = fs.readFileSync(this._filePath, { encoding: 'utf8' });
fileContents = this.fileSystem.readFileSync(this._filePath, 'utf8');
// Remember the length and hash for comparison purposes.
this._lastFileContentLength = fileContents.length;
@ -442,7 +446,7 @@ export class SourceFile {
diagSink.addError(`Source file could not be read`, getEmptyRange());
fileContents = '';
if (!fs.existsSync(this._filePath)) {
if (!this.fileSystem.existsSync(this._filePath)) {
this._isFileDeleted = true;
}
}
@ -521,8 +525,8 @@ export class SourceFile {
return true;
}
getDefinitionsForPosition(position: DiagnosticTextPosition,
evaluator: TypeEvaluator): DocumentTextRange[] | undefined {
getDefinitionsForPosition(position: Position,
evaluator: TypeEvaluator): DocumentRange[] | undefined {
// If we have no completed analysis job, there's nothing to do.
if (!this._parseResults) {
@ -530,11 +534,11 @@ export class SourceFile {
}
return DefinitionProvider.getDefinitionsForPosition(
this._parseResults, position, evaluator);
this._parseResults, position, evaluator);
}
getReferencesForPosition(position: DiagnosticTextPosition, includeDeclaration: boolean,
evaluator: TypeEvaluator): ReferencesResult | undefined {
getReferencesForPosition(position: Position, includeDeclaration: boolean,
evaluator: TypeEvaluator): ReferencesResult | undefined {
// If we have no completed analysis job, there's nothing to do.
if (!this._parseResults) {
@ -546,7 +550,7 @@ export class SourceFile {
}
addReferences(referencesResult: ReferencesResult, includeDeclaration: boolean,
evaluator: TypeEvaluator): void {
evaluator: TypeEvaluator): void {
// If we have no completed analysis job, there's nothing to do.
if (!this._parseResults) {
@ -569,7 +573,7 @@ export class SourceFile {
}
addSymbolsForDocument(symbolList: SymbolInformation[], evaluator: TypeEvaluator,
query?: string) {
query?: string) {
// If we have no completed analysis job, there's nothing to do.
if (!this._parseResults) {
@ -580,8 +584,8 @@ export class SourceFile {
this._filePath, this._parseResults, evaluator);
}
getHoverForPosition(position: DiagnosticTextPosition,
evaluator: TypeEvaluator): HoverResults | undefined {
getHoverForPosition(position: Position,
evaluator: TypeEvaluator): HoverResults | undefined {
// If this file hasn't been bound, no hover info is available.
if (this._isBindingNeeded || !this._parseResults) {
@ -592,8 +596,8 @@ export class SourceFile {
this._parseResults, position, evaluator);
}
getSignatureHelpForPosition(position: DiagnosticTextPosition,
importLookup: ImportLookup, evaluator: TypeEvaluator): SignatureHelpResults | undefined {
getSignatureHelpForPosition(position: Position,
importLookup: ImportLookup, evaluator: TypeEvaluator): SignatureHelpResults | undefined {
// If we have no completed analysis job, there's nothing to do.
if (!this._parseResults) {
@ -604,10 +608,10 @@ export class SourceFile {
this._parseResults, position, evaluator);
}
getCompletionsForPosition(position: DiagnosticTextPosition,
workspacePath: string, configOptions: ConfigOptions, importResolver: ImportResolver,
importLookup: ImportLookup, evaluator: TypeEvaluator,
moduleSymbolsCallback: () => ModuleSymbolMap): CompletionList | undefined {
getCompletionsForPosition(position: Position,
workspacePath: string, configOptions: ConfigOptions, importResolver: ImportResolver,
importLookup: ImportLookup, evaluator: TypeEvaluator,
moduleSymbolsCallback: () => ModuleSymbolMap): CompletionList | undefined {
// If we have no completed analysis job, there's nothing to do.
if (!this._parseResults) {
@ -630,10 +634,10 @@ export class SourceFile {
}
resolveCompletionItem(configOptions: ConfigOptions, importResolver: ImportResolver,
importLookup: ImportLookup, evaluator: TypeEvaluator,
moduleSymbolsCallback: () => ModuleSymbolMap, completionItem: CompletionItem) {
importLookup: ImportLookup, evaluator: TypeEvaluator,
moduleSymbolsCallback: () => ModuleSymbolMap, completionItem: CompletionItem) {
if (!this._parseResults || this._fileContents === undefined) {
if (!this._parseResults || this._fileContents === undefined) {
return;
}
@ -757,7 +761,7 @@ export class SourceFile {
}
private _buildFileInfo(configOptions: ConfigOptions, fileContents: string,
importLookup: ImportLookup, builtinsScope?: Scope) {
importLookup: ImportLookup, builtinsScope?: Scope) {
assert(this._parseResults !== undefined);
const analysisDiagnostics = new TextRangeDiagnosticSink(this._parseResults!.tokenizerOutput.lines);
@ -793,9 +797,9 @@ export class SourceFile {
}
private _resolveImports(importResolver: ImportResolver,
moduleImports: ModuleImport[],
execEnv: ExecutionEnvironment):
[ImportResult[], ImportResult?, string?, string?] {
moduleImports: ModuleImport[],
execEnv: ExecutionEnvironment):
[ImportResult[], ImportResult?, string?, string?] {
const imports: ImportResult[] = [];
@ -812,7 +816,7 @@ export class SourceFile {
// Avoid importing builtins from the builtins.pyi file itself.
if (builtinsImportResult.resolvedPaths.length === 0 ||
builtinsImportResult.resolvedPaths[0] !== this.getFilePath()) {
builtinsImportResult.resolvedPaths[0] !== this.getFilePath()) {
imports.push(builtinsImportResult);
} else {
builtinsImportResult = undefined;
@ -832,7 +836,7 @@ export class SourceFile {
// Avoid importing typing from the typing.pyi file itself.
let typingModulePath: string | undefined;
if (typingImportResult.resolvedPaths.length === 0 ||
typingImportResult.resolvedPaths[0] !== this.getFilePath()) {
typingImportResult.resolvedPaths[0] !== this.getFilePath()) {
imports.push(typingImportResult);
typingModulePath = typingImportResult.resolvedPaths[0];
}

4
server/src/analyzer/typeEvaluator.ts
View File

@ -17,8 +17,8 @@
import * as assert from 'assert';
import { DiagnosticLevel } from '../common/configOptions';
import { AddMissingOptionalToParamAction, Diagnostic, DiagnosticAddendum,
getEmptyRange } from '../common/diagnostic';
import { AddMissingOptionalToParamAction, Diagnostic, DiagnosticAddendum } from '../common/diagnostic';
import { getEmptyRange } from '../common/textRange';
import { DiagnosticRule } from '../common/diagnosticRules';
import { convertOffsetsToRange } from '../common/positionUtils';
import { PythonVersion } from '../common/pythonVersion';

24
server/src/analyzer/typeStubWriter.ts
View File

@ -8,13 +8,13 @@
* and analyzed python source file.
*/
import * as fs from 'fs';
import { ArgumentCategory, ArgumentNode, AssignmentNode, AugmentedAssignmentNode,
import {
ArgumentCategory, ArgumentNode, AssignmentNode, AugmentedAssignmentNode,
ClassNode, DecoratorNode, ExpressionNode, ForNode, FunctionNode, IfNode,
ImportFromNode, ImportNode, ModuleNameNode, NameNode, ParameterCategory, ParameterNode,
ParseNode, ParseNodeType, StatementListNode, StringNode, TryNode,
TypeAnnotationNode, WhileNode, WithNode } from '../parser/parseNodes';
TypeAnnotationNode, WhileNode, WithNode
} from '../parser/parseNodes';
import * as AnalyzerNodeInfo from './analyzerNodeInfo';
import * as ParseTreeUtils from './parseTreeUtils';
import { ParseTreeWalker } from './parseTreeWalker';
@ -27,14 +27,14 @@ import { ClassType, isNoneOrNever, TypeCategory } from './types';
import * as TypeUtils from './typeUtils';
class TrackedImport {
constructor(public importName: string) {}
constructor(public importName: string) { }
isAccessed = false;
}
class TrackedImportAs extends TrackedImport {
constructor(importName: string, public alias: string | undefined,
public symbol: Symbol) {
public symbol: Symbol) {
super(importName);
}
@ -55,7 +55,7 @@ class TrackedImportFrom extends TrackedImport {
}
addSymbol(symbol: Symbol | undefined, name: string,
alias: string | undefined, isAccessed = false) {
alias: string | undefined, isAccessed = false) {
if (!this.symbols.find(s => s.name === name)) {
this.symbols.push({
@ -70,8 +70,8 @@ class TrackedImportFrom extends TrackedImport {
class ImportSymbolWalker extends ParseTreeWalker {
constructor(
private _accessedImportedSymbols: Map<string, boolean>,
private _treatStringsAsSymbols: boolean) {
private _accessedImportedSymbols: Map<string, boolean>,
private _treatStringsAsSymbols: boolean) {
super();
}
@ -116,7 +116,7 @@ export class TypeStubWriter extends ParseTreeWalker {
private _accessedImportedSymbols = new Map<string, boolean>();
constructor(private _typingsPath: string, private _sourceFile: SourceFile,
private _evaluator: TypeEvaluator) {
private _evaluator: TypeEvaluator) {
super();
// As a heuristic, we'll include all of the import statements
@ -556,7 +556,7 @@ export class TypeStubWriter extends ParseTreeWalker {
}
private _printExpression(node: ExpressionNode, isType = false,
treatStringsAsSymbols = false): string {
treatStringsAsSymbols = false): string {
const importSymbolWalker = new ImportSymbolWalker(
this._accessedImportedSymbols,
@ -640,6 +640,6 @@ export class TypeStubWriter extends ParseTreeWalker {
finalText += this._printTrackedImports();
finalText += this._typeStubText;
fs.writeFileSync(this._typingsPath, finalText, { encoding: 'utf8' });
this._sourceFile.fileSystem.writeFileSync(this._typingsPath, finalText, 'utf8');
}
}

256
server/src/common/collectionUtils.ts Normal file
View File

@ -0,0 +1,256 @@
/*
* collectionUtils.ts
* Copyright (c) Microsoft Corporation.
* Licensed under the MIT license.
*
* Various helper functions around collection/array
*/
import { Comparison, equateValues, compareValues, isArray } from "./core";
export const emptyArray: never[] = [] as never[];
export type EqualityComparer<T> = (a: T, b: T) => boolean;
export function contains<T>(array: readonly T[] | undefined, value: T, equalityComparer: EqualityComparer<T> = equateValues): boolean {
if (array) {
for (const v of array) {
if (equalityComparer(v, value)) {
return true;
}
}
}
return false;
}
/** Array that is only intended to be pushed to, never read. */
export interface Push<T> {
push(...values: T[]): void;
}
/**
* Appends a value to an array, returning the array.
*
* @param to The array to which `value` is to be appended. If `to` is `undefined`, a new array
* is created if `value` was appended.
* @param value The value to append to the array. If `value` is `undefined`, nothing is
* appended.
*/
export function append<TArray extends any[] | undefined, TValue extends NonNullable<TArray>[number] | undefined>(to: TArray, value: TValue): [undefined, undefined] extends [TArray, TValue] ? TArray : NonNullable<TArray>[number][];
export function append<T>(to: T[], value: T | undefined): T[];
export function append<T>(to: T[] | undefined, value: T): T[];
export function append<T>(to: T[] | undefined, value: T | undefined): T[] | undefined;
export function append<T>(to: Push<T>, value: T | undefined): void;
export function append<T>(to: T[], value: T | undefined): T[] | undefined {
if (value === undefined) return to;
if (to === undefined) return [value];
to.push(value);
return to;
}
/** Works like Array.prototype.find, returning `undefined` if no element satisfying the predicate is found. */
export function find<T, U extends T>(array: readonly T[], predicate: (element: T, index: number) => element is U): U | undefined;
export function find<T>(array: readonly T[], predicate: (element: T, index: number) => boolean): T | undefined;
export function find<T>(array: readonly T[], predicate: (element: T, index: number) => boolean): T | undefined {
for (let i = 0; i < array.length; i++) {
const value = array[i];
if (predicate(value, i)) {
return value;
}
}
return undefined;
}
/**
* Gets the actual offset into an array for a relative offset. Negative offsets indicate a
* position offset from the end of the array.
*/
function toOffset(array: readonly any[], offset: number) {
return offset < 0 ? array.length + offset : offset;
}
/**
* Appends a range of value to an array, returning the array.
*
* @param to The array to which `value` is to be appended. If `to` is `undefined`, a new array
* is created if `value` was appended.
* @param from The values to append to the array. If `from` is `undefined`, nothing is
* appended. If an element of `from` is `undefined`, that element is not appended.
* @param start The offset in `from` at which to start copying values.
* @param end The offset in `from` at which to stop copying values (non-inclusive).
*/
export function addRange<T>(to: T[], from: readonly T[] | undefined, start?: number, end?: number): T[];
export function addRange<T>(to: T[] | undefined, from: readonly T[] | undefined, start?: number, end?: number): T[] | undefined;
export function addRange<T>(to: T[] | undefined, from: readonly T[] | undefined, start?: number, end?: number): T[] | undefined {
if (from === undefined || from.length === 0) return to;
if (to === undefined) return from.slice(start, end);
start = start === undefined ? 0 : toOffset(from, start);
end = end === undefined ? from.length : toOffset(from, end);
for (let i = start; i < end && i < from.length; i++) {
if (from[i] !== undefined) {
to.push(from[i]);
}
}
return to;
}
export function insertAt<T>(array: T[], index: number, value: T) {
if (index === 0) {
array.unshift(value);
}
else if (index === array.length) {
array.push(value);
}
else {
for (let i = array.length; i > index; i--) {
array[i] = array[i - 1];
}
array[index] = value;
}
return array;
}
export type Comparer<T> = (a: T, b: T) => Comparison;
export interface SortedReadonlyArray<T> extends ReadonlyArray<T> {
" __sortedArrayBrand": any;
}
export interface SortedArray<T> extends Array<T> {
" __sortedArrayBrand": any;
}
/**
* Returns a new sorted array.
*/
export function cloneAndSort<T>(array: readonly T[], comparer?: Comparer<T>): SortedReadonlyArray<T> {
return (array.length === 0 ? array : array.slice().sort(comparer)) as SortedReadonlyArray<T>;
}
function selectIndex(_: unknown, i: number) {
return i;
}
function indicesOf(array: readonly unknown[]): number[] {
return array.map(selectIndex);
}
/**
* Stable sort of an array. Elements equal to each other maintain their relative position in the array.
*/
export function stableSort<T>(array: readonly T[], comparer: Comparer<T>): SortedReadonlyArray<T> {
const indices = indicesOf(array);
stableSortIndices(array, indices, comparer);
return indices.map(i => array[i]) as SortedArray<T> as SortedReadonlyArray<T>;
}
function stableSortIndices<T>(array: readonly T[], indices: number[], comparer: Comparer<T>) {
// sort indices by value then position
indices.sort((x, y) => comparer(array[x], array[y]) || compareValues(x, y));
}
export function map<T, U>(array: readonly T[], f: (x: T, i: number) => U): U[];
export function map<T, U>(array: readonly T[] | undefined, f: (x: T, i: number) => U): U[] | undefined;
export function map<T, U>(array: readonly T[] | undefined, f: (x: T, i: number) => U): U[] | undefined {
let result: U[] | undefined;
if (array) {
return array.map(f);
}
return result;
}
export function some<T>(array: readonly T[] | undefined): array is readonly T[];
export function some<T>(array: readonly T[] | undefined, predicate: (value: T) => boolean): boolean;
export function some<T>(array: readonly T[] | undefined, predicate?: (value: T) => boolean): boolean {
if (array) {
if (predicate) {
return array.some(predicate);
}
else {
return array.length > 0;
}
}
return false;
}
/**
* Iterates through `array` by index and performs the callback on each element of array until the callback
* returns a falsey value, then returns false.
* If no such value is found, the callback is applied to each element of array and `true` is returned.
*/
export function every<T>(array: readonly T[], callback: (element: T, index: number) => boolean): boolean {
if (array) {
return array.every(callback);
}
return true;
}
/**
* Performs a binary search, finding the index at which `value` occurs in `array`.
* If no such index is found, returns the 2's-complement of first index at which
* `array[index]` exceeds `value`.
* @param array A sorted array whose first element must be no larger than number
* @param value The value to be searched for in the array.
* @param keySelector A callback used to select the search key from `value` and each element of
* `array`.
* @param keyComparer A callback used to compare two keys in a sorted array.
* @param offset An offset into `array` at which to start the search.
*/
export function binarySearch<T, U>(array: readonly T[], value: T, keySelector: (v: T) => U, keyComparer: Comparer<U>, offset?: number): number {
return binarySearchKey(array, keySelector(value), keySelector, keyComparer, offset);
}
/**
* Performs a binary search, finding the index at which an object with `key` occurs in `array`.
* If no such index is found, returns the 2's-complement of first index at which
* `array[index]` exceeds `key`.
* @param array A sorted array whose first element must be no larger than number
* @param key The key to be searched for in the array.
* @param keySelector A callback used to select the search key from each element of `array`.
* @param keyComparer A callback used to compare two keys in a sorted array.
* @param offset An offset into `array` at which to start the search.
*/
export function binarySearchKey<T, U>(array: readonly T[], key: U, keySelector: (v: T) => U, keyComparer: Comparer<U>, offset?: number): number {
if (!some(array)) {
return -1;
}
let low = offset || 0;
let high = array.length - 1;
while (low <= high) {
const middle = low + ((high - low) >> 1);
const midKey = keySelector(array[middle]);
switch (keyComparer(midKey, key)) {
case Comparison.LessThan:
low = middle + 1;
break;
case Comparison.EqualTo:
return middle;
case Comparison.GreaterThan:
high = middle - 1;
break;
}
}
return ~low;
}
/**
* Flattens an array containing a mix of array or non-array elements.
*
* @param array The array to flatten.
*/
export function flatten<T>(array: T[][] | readonly (T | readonly T[] | undefined)[]): T[] {
const result = [];
for (const v of array) {
if (v) {
if (isArray(v)) {
addRange(result, v);
}
else {
result.push(v);
}
}
}
return result;
}

97
server/src/common/core.ts Normal file
View File

@ -0,0 +1,97 @@
/*
* core.ts
* Copyright (c) Microsoft Corporation.
* Licensed under the MIT license.
*
* Various misc code snippets that doesn't have any dependency to other user code.
* This is a place one can put code to break circular references between files.
* make sure this doesn't have any dependency to other files.
*/
export const enum Comparison {
LessThan = -1,
EqualTo = 0,
GreaterThan = 1
}
/**
* Safer version of `Function` which should not be called.
* Every function should be assignable to this, but this should not be assignable to every function.
*/
export type AnyFunction = (...args: never[]) => void;
/** Do nothing and return false */
export function returnFalse(): false { return false; }
/** Do nothing and return true */
export function returnTrue(): true { return true; }
/** Do nothing and return undefined */
export function returnUndefined(): undefined { return undefined; }
/** Returns its argument. */
export function identity<T>(x: T) { return x; }
/** Returns lower case string */
export function toLowerCase(x: string) { return x.toLowerCase(); }
export function equateValues<T>(a: T, b: T) { return a === b; }
export type GetCanonicalFileName = (fileName: string) => string;
export function compareComparableValues(a: string | undefined, b: string | undefined): Comparison;
export function compareComparableValues(a: number | undefined, b: number | undefined): Comparison;
export function compareComparableValues(a: string | number | undefined, b: string | number | undefined) {
return a === b ? Comparison.EqualTo :
a === undefined ? Comparison.LessThan :
b === undefined ? Comparison.GreaterThan :
a < b ? Comparison.LessThan :
Comparison.GreaterThan;
}
/**
* Compare two numeric values for their order relative to each other.
* To compare strings, use any of the `compareStrings` functions.
*/
export function compareValues(a: number | undefined, b: number | undefined): Comparison {
return compareComparableValues(a, b);
}
/**
* Tests whether a value is an array.
*/
export function isArray(value: any): value is readonly {}[] {
return Array.isArray ? Array.isArray(value) : value instanceof Array;
}
/**
* Tests whether a value is string
*/
export function isString(text: unknown): text is string {
return typeof text === 'string';
}
export function isNumber(x: unknown): x is number {
return typeof x === 'number';
}
const hasOwnProperty = Object.prototype.hasOwnProperty;
/**
* Type of objects whose values are all of the same type.
* The `in` and `for-in` operators can *not* be safely used,
* since `Object.prototype` may be modified by outside code.
*/
export interface MapLike<T> {
[index: string]: T;
}
/**
* Indicates whether a map-like contains an own property with the specified key.
*
* @param map A map-like.
* @param key A property key.
*/
export function hasProperty(map: MapLike<any>, key: string): boolean {
return hasOwnProperty.call(map, key);
}

105
server/src/common/debug.ts Normal file
View File

@ -0,0 +1,105 @@
/*
* debug.ts
* Copyright (c) Microsoft Corporation.
* Licensed under the MIT license.
*
* Various debug helper methods to show user friendly debugging info
*/
import { AnyFunction, compareValues, hasProperty } from "./core";
import { stableSort } from "./collectionUtils";
export function assert(expression: boolean, message?: string, verboseDebugInfo?: string | (() => string), stackCrawlMark?: AnyFunction): void {
if (!expression) {
if (verboseDebugInfo) {
message += "\r\nVerbose Debug Information: " + (typeof verboseDebugInfo === "string" ? verboseDebugInfo : verboseDebugInfo());
}
fail(message ? "False expression: " + message : "False expression.", stackCrawlMark || assert);
}
}
export function fail(message?: string, stackCrawlMark?: AnyFunction): never {
// debugger;
const e = new Error(message ? `Debug Failure. ${ message }` : "Debug Failure.");
if ((Error as any).captureStackTrace) {
(Error as any).captureStackTrace(e, stackCrawlMark || fail);
}
throw e;
}
export function assertDefined<T>(value: T | null | undefined, message?: string): T {
if (value === undefined || value === null) return fail(message);
return value;
}
export function assertEachDefined<T, A extends readonly T[]>(value: A, message?: string): A {
for (const v of value) {
assertDefined(v, message);
}
return value;
}
export function assertNever(member: never, message = "Illegal value:", stackCrawlMark?: AnyFunction): never {
const detail = JSON.stringify(member);
return fail(`${ message } ${ detail }`, stackCrawlMark || assertNever);
}
export function getFunctionName(func: AnyFunction) {
if (typeof func !== "function") {
return "";
}
else if (hasProperty(func, "name")) {
return (func as any).name;
}
else {
const text = Function.prototype.toString.call(func);
const match = /^function\s+([\w$]+)\s*\(/.exec(text);
return match ? match[1] : "";
}
}
/**
* Formats an enum value as a string for debugging and debug assertions.
*/
export function formatEnum(value = 0, enumObject: any, isFlags?: boolean) {
const members = getEnumMembers(enumObject);
if (value === 0) {
return members.length > 0 && members[0][0] === 0 ? members[0][1] : "0";
}
if (isFlags) {
let result = "";
let remainingFlags = value;
for (const [enumValue, enumName] of members) {
if (enumValue > value) {
break;
}
if (enumValue !== 0 && enumValue & value) {
result = `${ result }${ result ? "|" : "" }${ enumName }`;
remainingFlags &= ~enumValue;
}
}
if (remainingFlags === 0) {
return result;
}
}
else {
for (const [enumValue, enumName] of members) {
if (enumValue === value) {
return enumName;
}
}
}
return value.toString();
}
function getEnumMembers(enumObject: any) {
const result: [number, string][] = [];
for (const name in enumObject) {
const value = enumObject[name];
if (typeof value === "number") {
result.push([value, name]);
}
}
return stableSort<[number, string]>(result, (x, y) => compareValues(x[0], y[0]));
}

97
server/src/common/diagnostic.ts
View File

@ -1,14 +1,13 @@
/*
* diagnostics.ts
* Copyright (c) Microsoft Corporation.
* Licensed under the MIT license.
* Author: Eric Traut
*
* Class that represents errors and warnings.
*/
import { Position, Range } from 'vscode-languageserver';
* diagnostics.ts
* Copyright (c) Microsoft Corporation.
* Licensed under the MIT license.
* Author: Eric Traut
*
* Class that represents errors and warnings.
*/
import { CommandId } from '../definitions/commands';
import { Range } from './textRange';
export const enum DiagnosticCategory {
Error,
@ -16,78 +15,6 @@ export const enum DiagnosticCategory {
UnusedCode
}
export interface DiagnosticTextPosition {
// Both line and column are zero-based
line: number;
column: number;
}
export function comparePositions(a: DiagnosticTextPosition, b: DiagnosticTextPosition) {
if (a.line < b.line) {
return -1;
} else if (a.line > b.line) {
return 1;
} else if (a.column < b.column) {
return -1;
} else if (a.column > b.column) {
return 1;
}
return 0;
}
export function getEmptyPosition(): DiagnosticTextPosition {
return {
line: 0,
column: 0
};
}
export interface DiagnosticTextRange {
start: DiagnosticTextPosition;
end: DiagnosticTextPosition;
}
export function doRangesOverlap(a: DiagnosticTextRange, b: DiagnosticTextRange) {
if (comparePositions(b.start, a.end) >= 0) {
return false;
} else if (comparePositions(a.start, b.end) >= 0) {
return false;
}
return true;
}
export function doesRangeContain(range: DiagnosticTextRange, position: DiagnosticTextPosition) {
return comparePositions(range.start, position) >= 0 && comparePositions(range.end, position) <= 0;
}
export function rangesAreEqual(a: DiagnosticTextRange, b: DiagnosticTextRange) {
return comparePositions(a.start, b.start) === 0 && comparePositions(a.end, b.end) === 0;
}
export function getEmptyRange(): DiagnosticTextRange {
return {
start: getEmptyPosition(),
end: getEmptyPosition()
};
}
export function convertRange(range?: DiagnosticTextRange): Range {
if (!range) {
return Range.create(convertPosition(), convertPosition());
}
return Range.create(convertPosition(range.start), convertPosition(range.end));
}
export function convertPosition(position?: DiagnosticTextPosition): Position {
return !position ? Position.create(0, 0) : Position.create(position.line, position.column);
}
// Represents a range within a particular document.
export interface DocumentTextRange {
path: string;
range: DiagnosticTextRange;
}
export interface DiagnosticAction {
action: string;
}
@ -105,7 +32,7 @@ export interface AddMissingOptionalToParamAction extends DiagnosticAction {
export interface DiagnosticRelatedInfo {
message: string;
filePath: string;
range: DiagnosticTextRange;
range: Range;
}
// Represents a single error or warning.
@ -114,7 +41,9 @@ export class Diagnostic {
private _rule: string | undefined;
private _relatedInfo: DiagnosticRelatedInfo[] = [];
constructor(readonly category: DiagnosticCategory, readonly message: string, readonly range: DiagnosticTextRange) {}
constructor(readonly category: DiagnosticCategory, readonly message: string,
readonly range: Range) {
}
addAction(action: DiagnosticAction) {
if (this._actions === undefined) {
@ -136,7 +65,7 @@ export class Diagnostic {
return this._rule;
}
addRelatedInfo(message: string, filePath: string, range: DiagnosticTextRange) {
addRelatedInfo(message: string, filePath: string, range: Range) {
this._relatedInfo.push({ filePath, message, range });
}

10
server/src/common/diagnosticSink.ts
View File

@ -7,9 +7,9 @@
* Class that represents errors and warnings.
*/
import { Diagnostic, DiagnosticCategory, DiagnosticTextRange } from './diagnostic';
import { Diagnostic, DiagnosticCategory } from './diagnostic';
import { convertOffsetsToRange } from './positionUtils';
import { TextRange } from './textRange';
import { TextRange, Range } from './textRange';
import { TextRangeCollection } from './textRangeCollection';
// Represents a collection of diagnostics within a file.
@ -32,15 +32,15 @@ export class DiagnosticSink {
return prevDiagnostics;
}
addError(message: string, range: DiagnosticTextRange) {
addError(message: string, range: Range) {
return this.addDiagnostic(new Diagnostic(DiagnosticCategory.Error, message, range));
}
addWarning(message: string, range: DiagnosticTextRange) {
addWarning(message: string, range: Range) {
return this.addDiagnostic(new Diagnostic(DiagnosticCategory.Warning, message, range));
}
addUnusedCode(message: string, range: DiagnosticTextRange) {
addUnusedCode(message: string, range: Range) {
return this.addDiagnostic(new Diagnostic(DiagnosticCategory.UnusedCode, message, range));
}

4
server/src/common/editAction.ts
View File

@ -7,10 +7,10 @@
* Represents a single edit within a file.
*/
import { DiagnosticTextRange } from './diagnostic';
import { Range } from "./textRange";
export interface TextEditAction {
range: DiagnosticTextRange;
range: Range;
replacementText: string;
}

395
server/src/common/pathUtils.ts
View File

@ -7,9 +7,13 @@
* Pathname utility functions.
*/
import * as fs from 'fs';
import * as path from 'path';
import Char from 'typescript-char';
import { some } from './collectionUtils';
import { compareValues, Comparison, GetCanonicalFileName, identity } from './core';
import * as debug from './debug';
import { getStringComparer, equateStringsCaseInsensitive, equateStringsCaseSensitive, compareStringsCaseSensitive, compareStringsCaseInsensitive } from './stringUtils';
import { VirtualFileSystem } from './vfs';
import { URI } from 'vscode-uri';
export interface FileSpec {
@ -72,11 +76,15 @@ export function getPathComponents(pathString: string) {
rest.pop();
}
const components = [root, ...rest];
const reduced = [components[0]];
return reducePathComponents([root, ...rest]);
}
export function reducePathComponents(components: readonly string[]) {
if (!some(components)) return [];
// Reduce the path components by eliminating
// any '.' or '..'.
const reduced = [components[0]];
for (let i = 1; i < components.length; i++) {
const component = components[i];
if (!component || component === '.') {
@ -99,6 +107,13 @@ export function getPathComponents(pathString: string) {
return reduced;
}
export function combinePathComponents(components: string[]): string {
if (components.length === 0) return "";
const root = components[0] && ensureTrailingDirectorySeparator(components[0]);
return normalizeSlashes(root + components.slice(1).join(path.sep));
}
export function getRelativePath(dirPath: string, relativeTo: string) {
if (!dirPath.startsWith(ensureTrailingDirectorySeparator(relativeTo))) {
return undefined;
@ -116,7 +131,7 @@ export function getRelativePath(dirPath: string, relativeTo: string) {
}
// Creates a directory hierarchy for a path, starting from some ancestor path.
export function makeDirectories(dirPath: string, startingFromDirPath: string) {
export function makeDirectories(fs: VirtualFileSystem, dirPath: string, startingFromDirPath: string) {
if (!dirPath.startsWith(startingFromDirPath)) {
return;
}
@ -133,11 +148,44 @@ export function makeDirectories(dirPath: string, startingFromDirPath: string) {
}
}
export function getFileSize(fs: VirtualFileSystem, path: string) {
try {
const stat = fs.statSync(path);
if (stat.isFile()) {
return stat.size;
}
}
catch { /*ignore*/ }
return 0;
}
export function fileExists(fs: VirtualFileSystem, path: string): boolean {
return fileSystemEntryExists(fs, path, FileSystemEntryKind.File);
}
export function directoryExists(fs: VirtualFileSystem, path: string): boolean {
return fileSystemEntryExists(fs, path, FileSystemEntryKind.Directory);
}
export function normalizeSlashes(pathString: string): string {
const separatorRegExp = /[\\/]/g;
return pathString.replace(separatorRegExp, path.sep);
}
/**
* Combines and resolves paths. If a path is absolute, it replaces any previous path. Any
* `.` and `..` path components are resolved. Trailing directory separators are preserved.
*
* ```ts
* resolvePath("/path", "to", "file.ext") === "path/to/file.ext"
* resolvePath("/path", "to", "file.ext/") === "path/to/file.ext/"
* resolvePath("/path", "dir", "..", "to", "file.ext") === "path/to/file.ext"
* ```
*/
export function resolvePaths(path: string, ...paths: (string | undefined)[]): string {
return normalizePath(some(paths) ? combinePaths(path, ...paths) : normalizeSlashes(path));
}
export function combinePaths(pathString: string, ...paths: (string | undefined)[]): string {
if (pathString) {
pathString = normalizeSlashes(pathString);
@ -160,6 +208,194 @@ export function combinePaths(pathString: string, ...paths: (string | undefined)[
return pathString;
}
/**
* Compare two paths using the provided case sensitivity.
*/
export function comparePaths(a: string, b: string, ignoreCase?: boolean): Comparison;
export function comparePaths(a: string, b: string, currentDirectory: string, ignoreCase?: boolean): Comparison;
export function comparePaths(a: string, b: string, currentDirectory?: string | boolean, ignoreCase?: boolean) {
a = normalizePath(a);
b = normalizePath(b);
if (typeof currentDirectory === "string") {
a = combinePaths(currentDirectory, a);
b = combinePaths(currentDirectory, b);
}
else if (typeof currentDirectory === "boolean") {
ignoreCase = currentDirectory;
}
return comparePathsWorker(a, b, getStringComparer(ignoreCase));
}
/**
* Determines whether a `parent` path contains a `child` path using the provide case sensitivity.
*/
export function containsPath(parent: string, child: string, ignoreCase?: boolean): boolean;
export function containsPath(parent: string, child: string, currentDirectory: string, ignoreCase?: boolean): boolean;
export function containsPath(parent: string, child: string, currentDirectory?: string | boolean, ignoreCase?: boolean) {
if (typeof currentDirectory === "string") {
parent = combinePaths(currentDirectory, parent);
child = combinePaths(currentDirectory, child);
}
else if (typeof currentDirectory === "boolean") {
ignoreCase = currentDirectory;
}
if (parent === undefined || child === undefined) return false;
if (parent === child) return true;
const parentComponents = getPathComponents(parent);
const childComponents = getPathComponents(child);
if (childComponents.length < parentComponents.length) {
return false;
}
const componentEqualityComparer = ignoreCase ? equateStringsCaseInsensitive : equateStringsCaseSensitive;
for (let i = 0; i < parentComponents.length; i++) {
const equalityComparer = i === 0 ? equateStringsCaseInsensitive : componentEqualityComparer;
if (!equalityComparer(parentComponents[i], childComponents[i])) {
return false;
}
}
return true;
}
/**
* Changes the extension of a path to the provided extension.
*
* ```ts
* changeAnyExtension("/path/to/file.ext", ".js") === "/path/to/file.js"
* ```
*/
export function changeAnyExtension(path: string, ext: string): string;
/**
* Changes the extension of a path to the provided extension if it has one of the provided extensions.
*
* ```ts
* changeAnyExtension("/path/to/file.ext", ".js", ".ext") === "/path/to/file.js"
* changeAnyExtension("/path/to/file.ext", ".js", ".ts") === "/path/to/file.ext"
* changeAnyExtension("/path/to/file.ext", ".js", [".ext", ".ts"]) === "/path/to/file.js"
* ```
*/
export function changeAnyExtension(path: string, ext: string, extensions: string | readonly string[], ignoreCase: boolean): string;
export function changeAnyExtension(path: string, ext: string, extensions?: string | readonly string[], ignoreCase?: boolean): string {
const pathext = extensions !== undefined && ignoreCase !== undefined ? getAnyExtensionFromPath(path, extensions, ignoreCase) : getAnyExtensionFromPath(path);
return pathext ? path.slice(0, path.length - pathext.length) + (ext.startsWith(".") ? ext : "." + ext) : path;
}
/**
* Gets the file extension for a path.
*
* ```ts
* getAnyExtensionFromPath("/path/to/file.ext") === ".ext"
* getAnyExtensionFromPath("/path/to/file.ext/") === ".ext"
* getAnyExtensionFromPath("/path/to/file") === ""
* getAnyExtensionFromPath("/path/to.ext/file") === ""
* ```
*/
export function getAnyExtensionFromPath(path: string): string;
/**
* Gets the file extension for a path, provided it is one of the provided extensions.
*
* ```ts
* getAnyExtensionFromPath("/path/to/file.ext", ".ext", true) === ".ext"
* getAnyExtensionFromPath("/path/to/file.js", ".ext", true) === ""
* getAnyExtensionFromPath("/path/to/file.js", [".ext", ".js"], true) === ".js"
* getAnyExtensionFromPath("/path/to/file.ext", ".EXT", false) === ""
*/
export function getAnyExtensionFromPath(path: string, extensions: string | readonly string[], ignoreCase: boolean): string;
export function getAnyExtensionFromPath(path: string, extensions?: string | readonly string[], ignoreCase?: boolean): string {
// Retrieves any string from the final "." onwards from a base file name.
// Unlike extensionFromPath, which throws an exception on unrecognized extensions.
if (extensions) {
return getAnyExtensionFromPathWorker(stripTrailingDirectorySeparator(path), extensions, ignoreCase ? equateStringsCaseInsensitive : equateStringsCaseSensitive);
}
const baseFileName = getBaseFileName(path);
const extensionIndex = baseFileName.lastIndexOf(".");
if (extensionIndex >= 0) {
return baseFileName.substring(extensionIndex);
}
return "";
}
/**
* Returns the path except for its containing directory name.
* Semantics align with NodeJS's `path.basename` except that we support URL's as well.
*
* ```ts
* // POSIX
* getBaseFileName("/path/to/file.ext") === "file.ext"
* getBaseFileName("/path/to/") === "to"
* getBaseFileName("/") === ""
* // DOS
* getBaseFileName("c:/path/to/file.ext") === "file.ext"
* getBaseFileName("c:/path/to/") === "to"
* getBaseFileName("c:/") === ""
* getBaseFileName("c:") === ""
* ```
*/
export function getBaseFileName(pathString: string): string;
/**
* Gets the portion of a path following the last (non-terminal) separator (`/`).
* Semantics align with NodeJS's `path.basename` except that we support URL's as well.
* If the base name has any one of the provided extensions, it is removed.
*
* ```ts
* getBaseFileName("/path/to/file.ext", ".ext", true) === "file"
* getBaseFileName("/path/to/file.js", ".ext", true) === "file.js"
* getBaseFileName("/path/to/file.js", [".ext", ".js"], true) === "file"
* getBaseFileName("/path/to/file.ext", ".EXT", false) === "file.ext"
* ```
*/
export function getBaseFileName(pathString: string, extensions: string | readonly string[], ignoreCase: boolean): string;
export function getBaseFileName(pathString: string, extensions?: string | readonly string[], ignoreCase?: boolean) {
pathString = normalizeSlashes(pathString);
// if the path provided is itself the root, then it has not file name.
const rootLength = getRootLength(pathString);
if (rootLength === pathString.length) return "";
// return the trailing portion of the path starting after the last (non-terminal) directory
// separator but not including any trailing directory separator.
pathString = stripTrailingDirectorySeparator(pathString);
const name = pathString.slice(Math.max(getRootLength(pathString), pathString.lastIndexOf(path.sep) + 1));
const extension = extensions !== undefined && ignoreCase !== undefined ? getAnyExtensionFromPath(name, extensions, ignoreCase) : undefined;
return extension ? name.slice(0, name.length - extension.length) : name;
}
/**
* Gets a relative path that can be used to traverse between `from` and `to`.
*/
export function getRelativePathFromDirectory(from: string, to: string, ignoreCase: boolean): string;
/**
* Gets a relative path that can be used to traverse between `from` and `to`.
*/
export function getRelativePathFromDirectory(fromDirectory: string, to: string, getCanonicalFileName: GetCanonicalFileName): string;
export function getRelativePathFromDirectory(fromDirectory: string, to: string, getCanonicalFileNameOrIgnoreCase: GetCanonicalFileName | boolean) {
debug.assert((getRootLength(fromDirectory) > 0) === (getRootLength(to) > 0), "Paths must either both be absolute or both be relative");
const getCanonicalFileName = typeof getCanonicalFileNameOrIgnoreCase === "function" ? getCanonicalFileNameOrIgnoreCase : identity;
const ignoreCase = typeof getCanonicalFileNameOrIgnoreCase === "boolean" ? getCanonicalFileNameOrIgnoreCase : false;
const pathComponents = getPathComponentsRelativeTo(fromDirectory, to, ignoreCase ? equateStringsCaseInsensitive : equateStringsCaseSensitive, getCanonicalFileName);
return combinePathComponents(pathComponents);
}
/**
* Performs a case-sensitive comparison of two paths. Path roots are always compared case-insensitively.
*/
export function comparePathsCaseSensitive(a: string, b: string) {
return comparePathsWorker(a, b, compareStringsCaseSensitive);
}
/**
* Performs a case-insensitive comparison of two paths.
*/
export function comparePathsCaseInsensitive(a: string, b: string) {
return comparePathsWorker(a, b, compareStringsCaseInsensitive);
}
export function ensureTrailingDirectorySeparator(pathString: string): string {
if (!hasTrailingDirectorySeparator(pathString)) {
return pathString + path.sep;
@ -201,7 +437,7 @@ export function normalizePath(pathString: string): string {
return normalizeSlashes(path.normalize(pathString));
}
export function isDirectory(path: string): boolean {
export function isDirectory(fs: VirtualFileSystem, path: string): boolean {
let stat: any;
try {
stat = fs.statSync(path);
@ -212,7 +448,7 @@ export function isDirectory(path: string): boolean {
return stat.isDirectory();
}
export function isFile(path: string): boolean {
export function isFile(fs: VirtualFileSystem, path: string): boolean {
let stat: any;
try {
stat = fs.statSync(path);
@ -223,7 +459,7 @@ export function isFile(path: string): boolean {
return stat.isFile();
}
export function getFileSystemEntries(path: string): FileSystemEntries {
export function getFileSystemEntries(fs: VirtualFileSystem, path: string): FileSystemEntries {
try {
const entries = fs.readdirSync(path || '.').sort();
const files: string[] = [];
@ -265,13 +501,16 @@ export function getWildcardRegexPattern(rootPath: string, fileSpec: string): str
}
const pathComponents = getPathComponents(absolutePath);
const doubleAsteriskRegexFragment = `(/[^/.][^/]*)*?`;
const reservedCharacterPattern = /[^\w\s/]/g;
const escapedSeparator = getRegexEscapedSeparator();
const doubleAsteriskRegexFragment = `(${escapedSeparator}[^${escapedSeparator}.][^${escapedSeparator}]*)*?`;
const reservedCharacterPattern = new RegExp(`[^\\w\\s${escapedSeparator}]`, "g");
// Strip the directory separator from the root component.
if (pathComponents.length > 0) {
pathComponents[0] = stripTrailingDirectorySeparator(pathComponents[0]);
}
let regExPattern = '';
let firstComponent = true;
@ -280,15 +519,15 @@ export function getWildcardRegexPattern(rootPath: string, fileSpec: string): str
regExPattern += doubleAsteriskRegexFragment;
} else {
if (!firstComponent) {
component = path.sep + component;
component = escapedSeparator + component;
}
regExPattern += component.replace(
reservedCharacterPattern, match => {
if (match === '*') {
return '[^/]*';
return `[^${escapedSeparator}]*`;
} else if (match === '?') {
return '[^/]';
return `[^${escapedSeparator}]`;
} else {
return '\\' + match;
}
@ -340,8 +579,8 @@ export function getWildcardRoot(rootPath: string, fileSpec: string): string {
export function getFileSpec(rootPath: string, fileSpec: string): FileSpec {
let regExPattern = getWildcardRegexPattern(rootPath, fileSpec);
const escapedSeparator = path.sep === '/' ? '/' : '\\\\';
regExPattern = `^(${ regExPattern })($|${ escapedSeparator })`;
const escapedSeparator = getRegexEscapedSeparator();
regExPattern = `^(${regExPattern})($|${escapedSeparator})`;
const regExp = new RegExp(regExPattern);
const wildcardRoot = getWildcardRoot(rootPath, fileSpec);
@ -352,6 +591,134 @@ export function getFileSpec(rootPath: string, fileSpec: string): FileSpec {
};
}
export function getRegexEscapedSeparator() {
return path.sep === '/' ? '\\/' : '\\\\';
}
/**
* Determines whether a path is an absolute disk path (e.g. starts with `/`, or a dos path
* like `c:`, `c:\` or `c:/`).
*/
export function isRootedDiskPath(path: string) {
return getRootLength(path) > 0;
}
/**
* Determines whether a path consists only of a path root.
*/
export function isDiskPathRoot(path: string) {
const rootLength = getRootLength(path);
return rootLength > 0 && rootLength === path.length;
}
//// Path Comparisons
// check path for these segments: '', '.'. '..'
const relativePathSegmentRegExp = /(^|\/)\.{0,2}($|\/)/;
function comparePathsWorker(a: string, b: string, componentComparer: (a: string, b: string) => Comparison) {
if (a === b) return Comparison.EqualTo;
if (a === undefined) return Comparison.LessThan;
if (b === undefined) return Comparison.GreaterThan;
// NOTE: Performance optimization - shortcut if the root segments differ as there would be no
// need to perform path reduction.
const aRoot = a.substring(0, getRootLength(a));
const bRoot = b.substring(0, getRootLength(b));
const result = compareStringsCaseInsensitive(aRoot, bRoot);
if (result !== Comparison.EqualTo) {
return result;
}
// NOTE: Performance optimization - shortcut if there are no relative path segments in
// the non-root portion of the path
const aRest = a.substring(aRoot.length);
const bRest = b.substring(bRoot.length);
if (!relativePathSegmentRegExp.test(aRest) && !relativePathSegmentRegExp.test(bRest)) {
return componentComparer(aRest, bRest);
}
// The path contains a relative path segment. Normalize the paths and perform a slower component
// by component comparison.
const aComponents = getPathComponents(a);
const bComponents = getPathComponents(b);
const sharedLength = Math.min(aComponents.length, bComponents.length);
for (let i = 1; i < sharedLength; i++) {
const result = componentComparer(aComponents[i], bComponents[i]);
if (result !== Comparison.EqualTo) {
return result;
}
}
return compareValues(aComponents.length, bComponents.length);
}
function getAnyExtensionFromPathWorker(path: string, extensions: string | readonly string[], stringEqualityComparer: (a: string, b: string) => boolean) {
if (typeof extensions === "string") {
return tryGetExtensionFromPath(path, extensions, stringEqualityComparer) || "";
}
for (const extension of extensions) {
const result = tryGetExtensionFromPath(path, extension, stringEqualityComparer);
if (result) return result;
}
return "";
}
function tryGetExtensionFromPath(path: string, extension: string, stringEqualityComparer: (a: string, b: string) => boolean) {
if (!extension.startsWith(".")) extension = "." + extension;
if (path.length >= extension.length && path.charCodeAt(path.length - extension.length) === Char.Period) {
const pathExtension = path.slice(path.length - extension.length);
if (stringEqualityComparer(pathExtension, extension)) {
return pathExtension;
}
}
return undefined;
}
function getPathComponentsRelativeTo(from: string, to: string, stringEqualityComparer: (a: string, b: string) => boolean, getCanonicalFileName: GetCanonicalFileName) {
const fromComponents = getPathComponents(from);
const toComponents = getPathComponents(to);
let start: number;
for (start = 0; start < fromComponents.length && start < toComponents.length; start++) {
const fromComponent = getCanonicalFileName(fromComponents[start]);
const toComponent = getCanonicalFileName(toComponents[start]);
const comparer = start === 0 ? equateStringsCaseInsensitive : stringEqualityComparer;
if (!comparer(fromComponent, toComponent)) break;
}
if (start === 0) {
return toComponents;
}
const components = toComponents.slice(start);
const relative: string[] = [];
for (; start < fromComponents.length; start++) {
relative.push("..");
}
return ["", ...relative, ...components];
}
const enum FileSystemEntryKind {
File,
Directory,
}
function fileSystemEntryExists(fs: VirtualFileSystem, path: string, entryKind: FileSystemEntryKind): boolean {
try {
const stat = fs.statSync(path);
switch (entryKind) {
case FileSystemEntryKind.File: return stat.isFile();
case FileSystemEntryKind.Directory: return stat.isDirectory();
default: return false;
}
}
catch (e) {
return false;
}
}
export function convertUriToPath(uriString: string): string {
const uri = URI.parse(uriString);
let convertedPath = normalizePath(uri.path);

17
server/src/common/positionUtils.ts
View File

@ -10,17 +10,16 @@
import * as assert from 'assert';
import { DiagnosticTextPosition, DiagnosticTextRange } from './diagnostic';
import { TextRange } from './textRange';
import { Position, Range, TextRange } from './textRange';
import { TextRangeCollection } from './textRangeCollection';
// Translates a file offset into a line/column pair.
export function convertOffsetToPosition(offset: number, lines: TextRangeCollection<TextRange>): DiagnosticTextPosition {
export function convertOffsetToPosition(offset: number, lines: TextRangeCollection<TextRange>): Position {
// Handle the case where the file is empty.
if (lines.end === 0) {
return {
line: 0,
column: 0
character: 0
};
}
@ -35,24 +34,24 @@ export function convertOffsetToPosition(offset: number, lines: TextRangeCollecti
assert(lineRange !== undefined);
return {
line: itemIndex,
column: offset - lineRange.start
character: offset - lineRange.start
};
}
// Translates a start/end file offset into a pair of line/column positions.
export function convertOffsetsToRange(startOffset: number, endOffset: number,
lines: TextRangeCollection<TextRange>): DiagnosticTextRange {
lines: TextRangeCollection<TextRange>): Range {
const start = convertOffsetToPosition(startOffset, lines);
const end = convertOffsetToPosition(endOffset, lines);
return { start, end };
}
// Translates a position (line and col) into a file offset.
export function convertPositionToOffset(position: DiagnosticTextPosition,
lines: TextRangeCollection<TextRange>): number | undefined {
export function convertPositionToOffset(position: Position,
lines: TextRangeCollection<TextRange>): number | undefined {
if (position.line >= lines.count) {
return undefined;
}
return lines.getItemAt(position.line).start + position.column;
return lines.getItemAt(position.line).start + position.character;
}

79
server/src/common/stringUtils.ts
View File

@ -1,13 +1,14 @@
/*
* stringUtils.ts
* Copyright (c) Microsoft Corporation.
* Licensed under the MIT license.
* Author: Eric Traut
*
* Utility methods for manipulating and comparing strings.
*/
* stringUtils.ts
* Copyright (c) Microsoft Corporation.
* Licensed under the MIT license.
* Author: Eric Traut
*
* Utility methods for manipulating and comparing strings.
*/
import leven from 'leven';
import { compareComparableValues, Comparison } from './core';
// Determines how closely a typed string matches a symbol
// name. An exact match returns 1. A match that differs
@ -57,7 +58,69 @@ export function hashString(contents: string) {
let hash = 0;
for (let i = 0; i < contents.length; i++) {
hash = (hash << 5) - hash + contents.charCodeAt(i) | 0;
hash = ((hash << 5) - hash + contents.charCodeAt(i)) | 0;
}
return hash;
}
/**
* Compare two strings using a case-insensitive ordinal comparison.
*
* Ordinal comparisons are based on the difference between the unicode code points of both
* strings. Characters with multiple unicode representations are considered unequal. Ordinal
* comparisons provide predictable ordering, but place "a" after "B".
*
* Case-insensitive comparisons compare both strings one code-point at a time using the integer
* value of each code-point after applying `toUpperCase` to each string. We always map both
* strings to their upper-case form as some unicode characters do not properly round-trip to
* lowercase (such as `ẞ` (German sharp capital s)).
*/
export function compareStringsCaseInsensitive(a: string | undefined, b: string | undefined): Comparison {
return a === b
? Comparison.EqualTo
: a === undefined
? Comparison.LessThan
: b === undefined
? Comparison.GreaterThan
: compareComparableValues(a.toUpperCase(), b.toUpperCase());
}
/**
* Compare two strings using a case-sensitive ordinal comparison.
*
* Ordinal comparisons are based on the difference between the unicode code points of both
* strings. Characters with multiple unicode representations are considered unequal. Ordinal
* comparisons provide predictable ordering, but place "a" after "B".
*
* Case-sensitive comparisons compare both strings one code-point at a time using the integer
* value of each code-point.
*/
export function compareStringsCaseSensitive(a: string | undefined, b: string | undefined): Comparison {
return compareComparableValues(a, b);
}
export function getStringComparer(ignoreCase?: boolean) {
return ignoreCase ? compareStringsCaseInsensitive : compareStringsCaseSensitive;
}
/**
* Compare the equality of two strings using a case-insensitive ordinal comparison.
*
* Case-insensitive comparisons compare both strings one code-point at a time using the integer
* value of each code-point after applying `toUpperCase` to each string. We always map both
* strings to their upper-case form as some unicode characters do not properly round-trip to
* lowercase (such as `ẞ` (German sharp capital s)).
*/
export function equateStringsCaseInsensitive(a: string, b: string) {
return compareStringsCaseInsensitive(a, b) === Comparison.EqualTo;
}
/**
* Compare the equality of two strings using a case-sensitive ordinal comparison.
*
* Case-sensitive comparisons compare both strings one code-point at a time using the
* integer value of each code-point.
*/
export function equateStringsCaseSensitive(a: string, b: string) {
return compareStringsCaseSensitive(a, b) === Comparison.EqualTo;
}

72
server/src/common/textRange.ts
View File

@ -23,6 +23,16 @@ export namespace TextRange {
return { start, length };
}
export function fromBounds(start: number, end: number): TextRange {
if (start < 0) {
throw new Error('start must be non-negative');
}
if (start > end) {
throw new Error('end must be greater than or equal to start');
}
return create(start, end - start);
}
export function getEnd(range: TextRange): number {
return range.start + range.length;
}
@ -50,3 +60,65 @@ export namespace TextRange {
}
}
}
export interface Position {
// Both line and column are zero-based
line: number;
character: number;
}
export interface Range {
start: Position;
end: Position;
}
// Represents a range within a particular document.
export interface DocumentRange {
path: string;
range: Range;
}
export function comparePositions(a: Position, b: Position) {
if (a.line < b.line) {
return -1;
} else if (a.line > b.line) {
return 1;
} else if (a.character < b.character) {
return -1;
} else if (a.character > b.character) {
return 1;
}
return 0;
}
export function getEmptyPosition(): Position {
return {
line: 0,
character: 0
};
}
export function doRangesOverlap(a: Range, b: Range) {
if (comparePositions(b.start, a.end) >= 0) {
return false;
} else if (comparePositions(a.start, b.end) >= 0) {
return false;
}
return true;
}
export function doesRangeContain(range: Range, position: Position) {
return comparePositions(range.start, position) >= 0 &&
comparePositions(range.end, position) <= 0;
}
export function rangesAreEqual(a: Range, b: Range) {
return comparePositions(a.start, b.start) === 0 && comparePositions(a.end, b.end) === 0;
}
export function getEmptyRange(): Range {
return {
start: getEmptyPosition(),
end: getEmptyPosition()
};
}

131
server/src/common/vfs.ts Normal file
View File

@ -0,0 +1,131 @@
/*
* vfs.ts
* Copyright (c) Microsoft Corporation.
* Licensed under the MIT license.
*
* Defines virtual file system interface that our code will operate upon and
* factory method to expose real file system as virtual file system
*/
/* eslint-disable no-dupe-class-members */
// * NOTE * except tests, this should be only file that import "fs"
import * as fs from 'fs';
import * as chokidar from 'chokidar';
import { ConsoleInterface, NullConsole } from './console';
export type Listener = (eventName: 'add' | 'addDir' | 'change' | 'unlink' | 'unlinkDir', path: string, stats?: Stats) => void;
export interface FileWatcher {
close(): void;
}
export interface Stats {
size: number;
isFile(): boolean;
isDirectory(): boolean;
isBlockDevice(): boolean;
isCharacterDevice(): boolean;
isSymbolicLink(): boolean;
isFIFO(): boolean;
isSocket(): boolean;
}
export interface VirtualFileSystem {
existsSync(path: string): boolean;
mkdirSync(path: string): void;
chdir(path: string): void;
readdirSync(path: string): string[];
readFileSync(path: string, encoding?: null): Buffer;
readFileSync(path: string, encoding: string): string;
readFileSync(path: string, encoding?: string | null): string | Buffer;
writeFileSync(path: string, data: string | Buffer, encoding: string | null): void;
statSync(path: string): Stats;
unlinkSync(path: string): void;
realpathSync(path: string): string;
getModulePath(): string;
createFileSystemWatcher(paths: string[], event: 'all', listener: Listener): FileWatcher;
}
/**
* expose real file system as virtual file system
* @param console console to log messages
*/
export function createFromRealFileSystem(console?: ConsoleInterface): VirtualFileSystem {
return new FileSystem(console ?? new NullConsole());
}
const _isMacintosh = process.platform === 'darwin';
const _isLinux = process.platform === 'linux';
class FileSystem implements VirtualFileSystem {
constructor(private _console: ConsoleInterface) {
}
public existsSync(path: string) { return fs.existsSync(path) }
public mkdirSync(path: string) { fs.mkdirSync(path); }
public chdir(path: string) { process.chdir(path); }
public readdirSync(path: string) { return fs.readdirSync(path); }
public readFileSync(path: string, encoding?: null): Buffer;
public readFileSync(path: string, encoding: string): string;
public readFileSync(path: string, encoding?: string | null): Buffer | string;
public readFileSync(path: string, encoding: string | null = null) { return fs.readFileSync(path, { encoding: encoding }); }
public writeFileSync(path: string, data: string | Buffer, encoding: string | null) { fs.writeFileSync(path, data, { encoding: encoding }); }
public statSync(path: string) { return fs.statSync(path); }
public unlinkSync(path: string) { return fs.unlinkSync(path); }
public realpathSync(path: string) { return fs.realpathSync(path); }
public getModulePath(): string {
// The entry point to the tool should have set the __rootDirectory
// global variable to point to the directory that contains the
// typeshed-fallback directory.
return (global as any).__rootDirectory;
}
public createFileSystemWatcher(paths: string[], event: 'all', listener: Listener): FileWatcher {
return this._createBaseFileSystemWatcher(paths).on(event, listener);
}
private _createBaseFileSystemWatcher(paths: string[]): chokidar.FSWatcher {
// The following options are copied from VS Code source base. It also
// uses chokidar for its file watching.
const watcherOptions: chokidar.WatchOptions = {
ignoreInitial: true,
ignorePermissionErrors: true,
followSymlinks: true, // this is the default of chokidar and supports file events through symlinks
interval: 1000, // while not used in normal cases, if any error causes chokidar to fallback to polling, increase its intervals
binaryInterval: 1000,
disableGlobbing: true // fix https://github.com/Microsoft/vscode/issues/4586
};
if (_isMacintosh) {
// Explicitly disable on MacOS because it uses up large amounts of memory
// and CPU for large file hierarchies, resulting in instability and crashes.
watcherOptions.usePolling = false;
}
const excludes: string[] = [];
if (_isMacintosh || _isLinux) {
if (paths.some(path => path === '' || path === '/')) {
excludes.push('/dev/**');
if (_isLinux) {
excludes.push('/proc/**', '/sys/**');
}
}
}
watcherOptions.ignored = excludes;
const watcher = chokidar.watch(paths, watcherOptions);
watcher.on('error', _ => {
this._console.log('Error returned from file system watcher.');
});
// Detect if for some reason the native watcher library fails to load
if (_isMacintosh && !watcher.options.useFsEvents) {
this._console.log('Watcher could not use native fsevents library. File system watcher disabled.');
}
return watcher;
}
}

74
server/src/languageServerBase.ts
View File

@ -15,13 +15,15 @@ import {
import { AnalyzerService } from './analyzer/service';
import { CommandLineOptions } from './common/commandLineOptions';
import {
AddMissingOptionalToParamAction, convertRange, CreateTypeStubFileAction,
Diagnostic as AnalyzerDiagnostic, DiagnosticCategory, DiagnosticTextPosition, DiagnosticTextRange
AddMissingOptionalToParamAction, CreateTypeStubFileAction,
Diagnostic as AnalyzerDiagnostic, DiagnosticCategory
} from './common/diagnostic';
import './common/extensions';
import { combinePaths, getDirectoryPath, normalizePath, convertUriToPath, convertPathToUri } from './common/pathUtils';
import { combinePaths, convertPathToUri, convertUriToPath, getDirectoryPath, normalizePath } from './common/pathUtils';
import { CommandId } from './definitions/commands';
import { CompletionItemData } from './languageService/completionProvider';
import { Range, Position } from './common/textRange';
import { createFromRealFileSystem, VirtualFileSystem } from './common/vfs';
export interface ServerSettings {
venvPath?: string;
@ -43,6 +45,7 @@ export interface WorkspaceServiceInstance {
export abstract class LanguageServerBase {
// Create a connection for the server. The connection uses Node's IPC as a transport
private _connection: IConnection = createConnection(new IPCMessageReader(process), new IPCMessageWriter(process));
private _fs: VirtualFileSystem;
// Create a simple text document manager. The text document manager
// supports full document sync only.
private _documents: TextDocuments = new TextDocuments();
@ -55,7 +58,9 @@ export abstract class LanguageServerBase {
private _defaultWorkspacePath = '<default>';
constructor(private _productName: string, rootDirectory?: string) {
this._connection.console.log(`${_productName} language server starting`);
this._connection.console.log(`${ _productName } language server starting`);
// virtual file system to be used. initialized to real file system by default. but can't be overritten
this._fs = createFromRealFileSystem(this._connection.console);
// Stash the base directory into a global variable.
(global as any).__rootDirectory = rootDirectory ? rootDirectory : getDirectoryPath(__dirname);
// Make the text document manager listen on the connection
@ -86,8 +91,8 @@ export abstract class LanguageServerBase {
// Creates a service instance that's used for analyzing a
// program within a workspace.
private _createAnalyzerService(name: string): AnalyzerService {
this._connection.console.log(`Starting service instance "${name}"`);
const service = new AnalyzerService(name, this._connection.console);
this._connection.console.log(`Starting service instance "${ name }"`);
const service = new AnalyzerService(name, this._fs, this._connection.console);
// Don't allow the analysis engine to go too long without
// reporting results. This will keep it responsive.
@ -116,7 +121,7 @@ export abstract class LanguageServerBase {
const fileOrFiles = results.filesRequiringAnalysis !== 1 ? 'files' : 'file';
this._connection.sendNotification('pyright/reportProgress',
`${results.filesRequiringAnalysis} ${fileOrFiles} to analyze`);
`${ results.filesRequiringAnalysis } ${ fileOrFiles } to analyze`);
}
} else {
if (this._isDisplayingProgress) {
@ -135,7 +140,7 @@ export abstract class LanguageServerBase {
private _createTypeStubService(importName: string): AnalyzerService {
this._connection.console.log('Starting type stub service instance');
const service = new AnalyzerService('Type stub', this._connection.console);
const service = new AnalyzerService('Type stub', this._fs, this._connection.console);
service.setMaxAnalysisDuration({
openFilesTimeInMs: 500,
@ -273,14 +278,14 @@ export abstract class LanguageServerBase {
const filePath = convertUriToPath(params.textDocument.uri);
const workspace = this._getWorkspaceForFile(filePath);
if (!workspace.disableLanguageServices) {
const range: DiagnosticTextRange = {
const range: Range = {
start: {
line: params.range.start.line,
column: params.range.start.character
character: params.range.start.character
},
end: {
line: params.range.end.line,
column: params.range.end.character
character: params.range.end.character
}
};
@ -330,9 +335,9 @@ export abstract class LanguageServerBase {
const filePath = convertUriToPath(params.textDocument.uri);
const position: DiagnosticTextPosition = {
const position: Position = {
line: params.position.line,
column: params.position.character
character: params.position.character
};
const workspace = this._getWorkspaceForFile(filePath);
@ -344,15 +349,15 @@ export abstract class LanguageServerBase {
return undefined;
}
return locations.map(loc =>
Location.create(convertPathToUri(loc.path), convertRange(loc.range)));
Location.create(convertPathToUri(loc.path), loc.range));
});
this._connection.onReferences(params => {
const filePath = convertUriToPath(params.textDocument.uri);
const position: DiagnosticTextPosition = {
const position: Position = {
line: params.position.line,
column: params.position.character
character: params.position.character
};
const workspace = this._getWorkspaceForFile(filePath);
@ -365,7 +370,7 @@ export abstract class LanguageServerBase {
return undefined;
}
return locations.map(loc =>
Location.create(convertPathToUri(loc.path), convertRange(loc.range)));
Location.create(convertPathToUri(loc.path), loc.range));
});
this._connection.onDocumentSymbol(params => {
@ -399,9 +404,9 @@ export abstract class LanguageServerBase {
this._connection.onHover(params => {
const filePath = convertUriToPath(params.textDocument.uri);
const position: DiagnosticTextPosition = {
const position: Position = {
line: params.position.line,
column: params.position.character
character: params.position.character
};
const workspace = this._getWorkspaceForFile(filePath);
@ -422,16 +427,16 @@ export abstract class LanguageServerBase {
kind: MarkupKind.Markdown,
value: markupString
},
range: convertRange(hoverResults.range)
range: hoverResults.range
};
});
this._connection.onSignatureHelp(params => {
const filePath = convertUriToPath(params.textDocument.uri);
const position: DiagnosticTextPosition = {
const position: Position = {
line: params.position.line,
column: params.position.character
character: params.position.character
};
const workspace = this._getWorkspaceForFile(filePath);
@ -466,9 +471,9 @@ export abstract class LanguageServerBase {
this._connection.onCompletion(params => {
const filePath = convertUriToPath(params.textDocument.uri);
const position: DiagnosticTextPosition = {
const position: Position = {
line: params.position.line,
column: params.position.character
character: params.position.character
};
const workspace = this._getWorkspaceForFile(filePath);
@ -505,9 +510,9 @@ export abstract class LanguageServerBase {
this._connection.onRenameRequest(params => {
const filePath = convertUriToPath(params.textDocument.uri);
const position: DiagnosticTextPosition = {
const position: Position = {
line: params.position.line,
column: params.position.character
character: params.position.character
};
const workspace = this._getWorkspaceForFile(filePath);
@ -531,7 +536,7 @@ export abstract class LanguageServerBase {
}
const textEdit: TextEdit = {
range: convertRange(editAction.range),
range: editAction.range,
newText: editAction.replacementText
};
edits.changes![uri].push(textEdit);
@ -609,7 +614,7 @@ export abstract class LanguageServerBase {
const edits: TextEdit[] = [];
editActions.forEach(editAction => {
edits.push({
range: convertRange(editAction.range),
range: editAction.range,
newText: editAction.replacementText
});
});
@ -626,7 +631,7 @@ export abstract class LanguageServerBase {
// Allocate a temporary pseudo-workspace to perform this job.
const workspace: WorkspaceServiceInstance = {
workspaceName: `Create Type Stub ${importName}`,
workspaceName: `Create Type Stub ${ importName }`,
rootPath: workspaceRoot,
rootUri: convertPathToUri(workspaceRoot),
serviceInstance: service,
@ -638,7 +643,7 @@ export abstract class LanguageServerBase {
try {
service.writeTypeStub();
service.dispose();
const infoMessage = `Type stub was successfully created for '${importName}'.`;
const infoMessage = `Type stub was successfully created for '${ importName }'.`;
this._connection.window.showInformationMessage(infoMessage);
this._handlePostCreateTypeStub();
} catch (err) {
@ -646,7 +651,7 @@ export abstract class LanguageServerBase {
if (err instanceof Error) {
errMessage = ': ' + err.message;
}
errMessage = `An error occurred when creating type stub for '${importName}'` +
errMessage = `An error occurred when creating type stub for '${ importName }'` +
errMessage;
this._connection.console.error(errMessage);
this._connection.window.showErrorMessage(errMessage);
@ -740,10 +745,10 @@ export abstract class LanguageServerBase {
let source = this._productName;
const rule = diag.getRule();
if (rule) {
source = `${source} (${rule})`;
source = `${ source } (${ rule })`;
}
const vsDiag = Diagnostic.create(convertRange(diag.range), diag.message, severity,
const vsDiag = Diagnostic.create(diag.range, diag.message, severity,
undefined, source);
if (diag.category === DiagnosticCategory.UnusedCode) {
@ -755,8 +760,7 @@ export abstract class LanguageServerBase {
if (relatedInfo.length > 0) {
vsDiag.relatedInformation = relatedInfo.map(info => {
return DiagnosticRelatedInformation.create(
Location.create(convertPathToUri(info.filePath),
convertRange(info.range)),
Location.create(convertPathToUri(info.filePath), info.range),
info.message
);
});

22
server/src/languageService/completionProvider.ts
View File

@ -26,7 +26,7 @@ import { TypeEvaluator, CallSignatureInfo } from '../analyzer/typeEvaluator';
import { FunctionType, TypeCategory, ClassType, Type } from '../analyzer/types';
import { doForSubtypes, getMembersForClass, getMembersForModule } from '../analyzer/typeUtils';
import { ConfigOptions } from '../common/configOptions';
import { comparePositions, DiagnosticTextPosition } from '../common/diagnostic';
import { comparePositions, Position } from '../common/textRange';
import { TextEditAction } from '../common/editAction';
import { combinePaths, getDirectoryPath, getFileName, stripFileExtension } from '../common/pathUtils';
import { convertOffsetToPosition, convertPositionToOffset } from '../common/positionUtils';
@ -125,7 +125,7 @@ enum SortCategory {
export interface CompletionItemData {
filePath: string;
workspacePath: string;
position: DiagnosticTextPosition;
position: Position;
autoImportText?: string;
symbolId?: number;
}
@ -156,7 +156,7 @@ export class CompletionProvider {
private _parseResults: ParseResults,
private _fileContents: string,
private _importResolver: ImportResolver,
private _position: DiagnosticTextPosition,
private _position: Position,
private _filePath: string,
private _configOptions: ConfigOptions,
private _importLookup: ImportLookup,
@ -206,8 +206,8 @@ export class CompletionProvider {
// Get the text on that line prior to the insertion point.
const lineTextRange = this._parseResults.tokenizerOutput.lines.getItemAt(this._position.line);
const textOnLine = this._fileContents.substr(lineTextRange.start, lineTextRange.length);
const priorText = textOnLine.substr(0, this._position.column);
const postText = textOnLine.substr(this._position.column);
const priorText = textOnLine.substr(0, this._position.character);
const postText = textOnLine.substr(this._position.character);
const priorWordIndex = priorText.search(/\w+$/);
const priorWord = priorWordIndex >= 0 ? priorText.substr(priorWordIndex) : '';
@ -433,8 +433,8 @@ export class CompletionProvider {
const isSimilar = StringUtils.computeCompletionSimilarity(partialName.value, name) > similarityLimit;
if (isSimilar) {
const range: Range = {
start: { line: this._position.line, character: this._position.column - partialName.length },
end: { line: this._position.line, character: this._position.column }
start: { line: this._position.line, character: this._position.character - partialName.length },
end: { line: this._position.line, character: this._position.character }
};
const methodSignature = this._printMethodSignature(decl.node) + ':';
@ -745,14 +745,14 @@ export class CompletionProvider {
completionItem.kind = CompletionItemKind.Text;
completionItem.sortText = this._makeSortText(SortCategory.LiteralValue, valueWithQuotes);
let rangeStartCol = this._position.column;
let rangeStartCol = this._position.character;
if (priorString !== undefined) {
rangeStartCol -= priorString.length + 1;
}
// If the text after the insertion point is the closing quote,
// replace it.
let rangeEndCol = this._position.column;
let rangeEndCol = this._position.character;
if (postText !== undefined) {
if (postText.startsWith(quoteCharacter)) {
rangeEndCol++;
@ -1213,8 +1213,8 @@ export class CompletionProvider {
completionItem.additionalTextEdits = additionalTextEdits.map(te => {
const textEdit: TextEdit = {
range: {
start: { line: te.range.start.line, character: te.range.start.column },
end: { line: te.range.end.line, character: te.range.end.column }
start: { line: te.range.start.line, character: te.range.start.character },
end: { line: te.range.end.line, character: te.range.end.character }
},
newText: te.replacementText
};

10
server/src/languageService/definitionProvider.ts
View File

@ -12,14 +12,14 @@
import * as ParseTreeUtils from '../analyzer/parseTreeUtils';
import { TypeEvaluator } from '../analyzer/typeEvaluator';
import { DiagnosticTextPosition, DocumentTextRange, rangesAreEqual } from '../common/diagnostic';
import { Position, DocumentRange, rangesAreEqual } from '../common/textRange';
import { convertPositionToOffset } from '../common/positionUtils';
import { ParseNodeType } from '../parser/parseNodes';
import { ParseResults } from '../parser/parser';
export class DefinitionProvider {
static getDefinitionsForPosition(parseResults: ParseResults, position: DiagnosticTextPosition,
evaluator: TypeEvaluator): DocumentTextRange[] | undefined {
static getDefinitionsForPosition(parseResults: ParseResults, position: Position,
evaluator: TypeEvaluator): DocumentRange[] | undefined {
const offset = convertPositionToOffset(position, parseResults.tokenizerOutput.lines);
if (offset === undefined) {
@ -31,7 +31,7 @@ export class DefinitionProvider {
return undefined;
}
const definitions: DocumentTextRange[] = [];
const definitions: DocumentRange[] = [];
if (node.nodeType === ParseNodeType.Name) {
const declarations = evaluator.getDeclarationsForNameNode(node);
@ -51,7 +51,7 @@ export class DefinitionProvider {
return definitions.length > 0 ? definitions : undefined;
}
private static _addIfUnique(definitions: DocumentTextRange[], itemToAdd: DocumentTextRange) {
private static _addIfUnique(definitions: DocumentRange[], itemToAdd: DocumentRange) {
for (const def of definitions) {
if (def.path === itemToAdd.path && rangesAreEqual(def.range, itemToAdd.range)) {
return;

37
server/src/languageService/documentSymbolProvider.ts
View File

@ -8,8 +8,7 @@
* source file document.
*/
import { DocumentSymbol, Location, Position, Range, SymbolInformation,
SymbolKind } from 'vscode-languageserver';
import { DocumentSymbol, Location, SymbolInformation, SymbolKind } from 'vscode-languageserver';
import { URI } from 'vscode-uri';
import * as AnalyzerNodeInfo from '../analyzer/analyzerNodeInfo';
@ -19,7 +18,6 @@ import { ParseTreeWalker } from '../analyzer/parseTreeWalker';
import { getLastTypedDeclaredForSymbol } from '../analyzer/symbolUtils';
import { TypeEvaluator } from '../analyzer/typeEvaluator';
import { isProperty } from '../analyzer/typeUtils';
import { DiagnosticTextPosition, DiagnosticTextRange } from '../common/diagnostic';
import { convertOffsetsToRange } from '../common/positionUtils';
import * as StringUtils from '../common/stringUtils';
import { ClassNode, FunctionNode, ListComprehensionNode, ModuleNode, ParseNode } from '../parser/parseNodes';
@ -37,7 +35,7 @@ class FindSymbolTreeWalker extends ParseTreeWalker {
private _evaluator: TypeEvaluator;
constructor(filePath: string, parseResults: ParseResults, symbolInfoResults: SymbolInformation[],
query: string | undefined, evaluator: TypeEvaluator) {
query: string | undefined, evaluator: TypeEvaluator) {
super();
this._filePath = filePath;
@ -103,7 +101,7 @@ class FindSymbolTreeWalker extends ParseTreeWalker {
}
private _addSymbolInformationFromDeclaration(name: string, declaration: Declaration,
containerName?: string) {
containerName?: string) {
if (declaration.path !== this._filePath) {
return;
@ -123,7 +121,7 @@ class FindSymbolTreeWalker extends ParseTreeWalker {
const location: Location = {
uri: URI.file(this._filePath).toString(),
range: convertRange(declaration.range)
range: declaration.range
};
const symbolKind = getSymbolKind(name, declaration, this._evaluator);
@ -193,18 +191,9 @@ function getSymbolKind(name: string, declaration: Declaration, evaluator: TypeEv
return symbolKind;
}
function convertRange(range: DiagnosticTextRange): Range {
return Range.create(convertPosition(range.start),
convertPosition(range.end));
}
function convertPosition(position: DiagnosticTextPosition): Position {
return Position.create(position.line, position.column);
}
function getDocumentSymbolsRecursive(node: AnalyzerNodeInfo.ScopedNode,
docSymbolResults: DocumentSymbol[], parseResults: ParseResults,
evaluator: TypeEvaluator) {
docSymbolResults: DocumentSymbol[], parseResults: ParseResults,
evaluator: TypeEvaluator) {
const scope = AnalyzerNodeInfo.getScope(node);
if (!scope) {
@ -230,8 +219,8 @@ function getDocumentSymbolsRecursive(node: AnalyzerNodeInfo.ScopedNode,
}
function getDocumentSymbolRecursive(name: string, declaration: Declaration,
evaluator: TypeEvaluator, parseResults: ParseResults,
docSymbolResults: DocumentSymbol[]) {
evaluator: TypeEvaluator, parseResults: ParseResults,
docSymbolResults: DocumentSymbol[]) {
if (declaration.type === DeclarationType.Alias) {
return;
@ -242,19 +231,19 @@ function getDocumentSymbolRecursive(name: string, declaration: Declaration,
return;
}
const selectionRange = convertRange(declaration.range);
const selectionRange = declaration.range;
let range = selectionRange;
const children: DocumentSymbol[] = [];
if (declaration.type === DeclarationType.Class ||
declaration.type === DeclarationType.Function) {
declaration.type === DeclarationType.Function) {
getDocumentSymbolsRecursive(declaration.node, children, parseResults, evaluator);
const nameRange = convertOffsetsToRange(declaration.node.start,
declaration.node.name.start + declaration.node.length,
parseResults.tokenizerOutput.lines);
range = convertRange(nameRange);
range = nameRange;
}
const symbolInfo: DocumentSymbol = {
@ -270,7 +259,7 @@ function getDocumentSymbolRecursive(name: string, declaration: Declaration,
export class DocumentSymbolProvider {
static addSymbolsForDocument(symbolList: SymbolInformation[], query: string | undefined,
filePath: string, parseResults: ParseResults, evaluator: TypeEvaluator) {
filePath: string, parseResults: ParseResults, evaluator: TypeEvaluator) {
const symbolTreeWalker = new FindSymbolTreeWalker(filePath, parseResults,
symbolList, query, evaluator);
@ -278,7 +267,7 @@ export class DocumentSymbolProvider {
}
static addHierarchicalSymbolsForDocument(symbolList: DocumentSymbol[],
parseResults: ParseResults, evaluator: TypeEvaluator) {
parseResults: ParseResults, evaluator: TypeEvaluator) {
getDocumentSymbolsRecursive(parseResults.parseTree, symbolList, parseResults, evaluator);
}

6
server/src/languageService/hoverProvider.ts
View File

@ -15,7 +15,7 @@ import * as ParseTreeUtils from '../analyzer/parseTreeUtils';
import { TypeEvaluator } from '../analyzer/typeEvaluator';
import { Type, TypeCategory, UnknownType } from '../analyzer/types';
import { isProperty } from '../analyzer/typeUtils';
import { DiagnosticTextPosition, DiagnosticTextRange } from '../common/diagnostic';
import { Position, Range } from '../common/textRange';
import { convertOffsetToPosition, convertPositionToOffset } from '../common/positionUtils';
import { TextRange } from '../common/textRange';
import { NameNode, ParseNodeType } from '../parser/parseNodes';
@ -28,11 +28,11 @@ export interface HoverTextPart {
export interface HoverResults {
parts: HoverTextPart[];
range: DiagnosticTextRange;
range: Range;
}
export class HoverProvider {
static getHoverForPosition(parseResults: ParseResults, position: DiagnosticTextPosition,
static getHoverForPosition(parseResults: ParseResults, position: Position,
evaluator: TypeEvaluator): HoverResults | undefined {
const offset = convertPositionToOffset(position, parseResults.tokenizerOutput.lines);

4
server/src/languageService/importSorter.ts
View File

@ -10,7 +10,7 @@
import { ImportType } from '../analyzer/importResult';
import * as ImportStatementUtils from '../analyzer/importStatementUtils';
import { DiagnosticTextRange } from '../common/diagnostic';
import { Range } from '../common/textRange';
import { TextEditAction } from '../common/editAction';
import { convertOffsetToPosition } from '../common/positionUtils';
import { TextRange } from '../common/textRange';
@ -99,7 +99,7 @@ export class ImportSorter {
// If there are other blocks of import statements separated by other statements,
// we'll ignore these other blocks for now.
private _getPrimaryReplacementRange(statements: ImportStatementUtils.ImportStatement[]):
DiagnosticTextRange {
Range {
let statementLimit = statements.findIndex(s => s.followsNonImportStatement);
if (statementLimit < 0) {

6
server/src/languageService/referencesProvider.ts
View File

@ -14,7 +14,7 @@ import * as DeclarationUtils from '../analyzer/declarationUtils';
import * as ParseTreeUtils from '../analyzer/parseTreeUtils';
import { ParseTreeWalker } from '../analyzer/parseTreeWalker';
import { TypeEvaluator } from '../analyzer/typeEvaluator';
import { DiagnosticTextPosition, DocumentTextRange } from '../common/diagnostic';
import { Position, DocumentRange } from '../common/textRange';
import { convertOffsetToPosition, convertPositionToOffset } from '../common/positionUtils';
import { TextRange } from '../common/textRange';
import { NameNode, ParseNode, ParseNodeType } from '../parser/parseNodes';
@ -24,7 +24,7 @@ export interface ReferencesResult {
requiresGlobalSearch: boolean;
nodeAtOffset: ParseNode;
declarations: Declaration[];
locations: DocumentTextRange[];
locations: DocumentRange[];
}
class FindReferencesTreeWalker extends ParseTreeWalker {
@ -93,7 +93,7 @@ class FindReferencesTreeWalker extends ParseTreeWalker {
export class ReferencesProvider {
static getReferencesForPosition(parseResults: ParseResults, filePath: string,
position: DiagnosticTextPosition, includeDeclaration: boolean,
position: Position, includeDeclaration: boolean,
evaluator: TypeEvaluator): ReferencesResult | undefined {
const offset = convertPositionToOffset(position, parseResults.tokenizerOutput.lines);

8
server/src/languageService/signatureHelpProvider.ts
View File

@ -13,7 +13,7 @@ import { extractParameterDocumentation } from '../analyzer/docStringUtils';
import * as ParseTreeUtils from '../analyzer/parseTreeUtils';
import { TypeEvaluator } from '../analyzer/typeEvaluator';
import { FunctionType } from '../analyzer/types';
import { DiagnosticTextPosition } from '../common/diagnostic';
import { Position } from '../common/textRange';
import { convertPositionToOffset } from '../common/positionUtils';
import { ParseResults } from '../parser/parser';
@ -36,9 +36,9 @@ export interface SignatureHelpResults {
}
export class SignatureHelpProvider {
static getSignatureHelpForPosition(parseResults: ParseResults, position: DiagnosticTextPosition,
evaluator: TypeEvaluator):
SignatureHelpResults | undefined {
static getSignatureHelpForPosition(parseResults: ParseResults, position: Position,
evaluator: TypeEvaluator):
SignatureHelpResults | undefined {
const offset = convertPositionToOffset(position, parseResults.tokenizerOutput.lines);
if (offset === undefined) {

18
server/src/pyright.ts
View File

@ -19,9 +19,11 @@ import * as process from 'process';
import { AnalyzerService } from './analyzer/service';
import { CommandLineOptions as PyrightCommandLineOptions } from './common/commandLineOptions';
import { NullConsole } from './common/console';
import { DiagnosticCategory, DiagnosticTextRange } from './common/diagnostic';
import { DiagnosticCategory } from './common/diagnostic';
import { Range } from './common/textRange';
import { FileDiagnostics } from './common/diagnosticSink';
import { combinePaths, normalizePath } from './common/pathUtils';
import { createFromRealFileSystem } from './common/vfs';
const toolName = 'pyright';
@ -43,7 +45,7 @@ interface PyrightJsonDiagnostic {
file: string;
severity: 'error' | 'warning';
message: string;
range: DiagnosticTextRange;
range: Range;
}
interface PyrightJsonSummary {
@ -149,8 +151,8 @@ function processArgs() {
const watch = args.watch !== undefined;
options.watch = watch;
const service = new AnalyzerService('<default>', args.outputjson ?
new NullConsole() : undefined);
const output = args.outputjson ? new NullConsole() : undefined;
const service = new AnalyzerService('<default>', createFromRealFileSystem(output), output);
service.setCompletionCallback(results => {
if (results.fatalErrorOccurred) {
@ -207,8 +209,8 @@ function processArgs() {
if (!watch) {
process.exit(
errorCount > 0 ?
ExitStatus.ErrorsReported :
ExitStatus.NoErrors);
ExitStatus.ErrorsReported :
ExitStatus.NoErrors);
} else {
console.log('Watching for file changes...');
}
@ -253,7 +255,7 @@ function printVersion() {
}
function reportDiagnosticsAsJson(fileDiagnostics: FileDiagnostics[], filesInProgram: number,
timeInSec: number): DiagnosticResult {
timeInSec: number): DiagnosticResult {
const report: PyrightJsonResults = {
version: getVersionString(),
@ -316,7 +318,7 @@ function reportDiagnosticsAsText(fileDiagnostics: FileDiagnostics[]): Diagnostic
let message = ' ';
if (diag.range) {
message += chalk.yellow(`${ diag.range.start.line + 1 }`) + ':' +
chalk.yellow(`${ diag.range.start.column + 1 }`) + ' - ';
chalk.yellow(`${ diag.range.start.character + 1 }`) + ' - ';
}
message += diag.category === DiagnosticCategory.Error ?

8
server/src/server.ts
View File

@ -1,8 +1,8 @@
/*
* server.ts
*
* Implements pyright language server.
*/
* server.ts
*
* Implements pyright language server.
*/
import { isArray } from 'util';
import { LanguageServerBase, ServerSettings, WorkspaceServiceInstance } from './languageServerBase';

14
server/src/tests/binder.test.ts
View File

@ -1,11 +1,11 @@
/*
* binder.test.ts
* Copyright (c) Microsoft Corporation.
* Licensed under the MIT license.
* Author: Eric Traut
*
* Unit tests for pyright name binder.
*/
* binder.test.ts
* Copyright (c) Microsoft Corporation.
* Licensed under the MIT license.
* Author: Eric Traut
*
* Unit tests for pyright name binder.
*/
import * as assert from 'assert';

50
server/src/tests/checker.test.ts
View File

@ -1,11 +1,11 @@
/*
* checker.test.ts
* Copyright (c) Microsoft Corporation.
* Licensed under the MIT license.
* Author: Eric Traut
*
* Unit tests for pyright type checker and type analyzer.
*/
* checker.test.ts
* Copyright (c) Microsoft Corporation.
* Licensed under the MIT license.
* Author: Eric Traut
*
* Unit tests for pyright type checker and type analyzer.
*/
import * as assert from 'assert';
@ -436,42 +436,6 @@ test('NamedTuples1', () => {
validateResults(analysisResults, 6);
});
test('DataClass1', () => {
const analysisResults = TestUtils.typeAnalyzeSampleFiles(['dataclass1.py']);
validateResults(analysisResults, 2);
});
test('DataClass3', () => {
const analysisResults = TestUtils.typeAnalyzeSampleFiles(['dataclass3.py']);
validateResults(analysisResults, 1);
});
test('DataClass4', () => {
const analysisResults = TestUtils.typeAnalyzeSampleFiles(['dataclass4.py']);
validateResults(analysisResults, 5);
});
test('DataClass5', () => {
const analysisResults = TestUtils.typeAnalyzeSampleFiles(['dataclass5.py']);
validateResults(analysisResults, 2);
});
test('DataClass6', () => {
const analysisResults = TestUtils.typeAnalyzeSampleFiles(['dataclass6.py']);
validateResults(analysisResults, 2);
});
test('DataClass7', () => {
const analysisResults = TestUtils.typeAnalyzeSampleFiles(['dataclass7.py']);
validateResults(analysisResults, 2);
});
test('AbstractClass1', () => {
const analysisResults = TestUtils.typeAnalyzeSampleFiles(['abstractClass1.py']);

147
server/src/tests/collectionUtils.test.ts Normal file
View File

@ -0,0 +1,147 @@
/*
* collectionUtils.test.ts
* Copyright (c) Microsoft Corporation.
* Licensed under the MIT license.
*/
import * as assert from 'assert';
import * as utils from '../common/collectionUtils';
import { compareValues, isArray } from '../common/core';
test('UtilsContainsDefault', () => {
const data = [1, 2, 3, 4, 5];
assert(utils.contains(data, 2));
});
test('UtilsContainsComparer', () => {
const data = [new D(1, "A"), new D(2, "B"), new D(3, "C"), new D(4, "D")]
assert(utils.contains(data, new D(1, "D"), (a, b) => a.Value == b.Value));
});
test('UtilsAppend', () => {
const data: number[] = [];
assert.deepEqual(utils.append(data, 1), [1]);
});
test('UtilsAppendUndefined', () => {
const data = undefined;
assert.deepEqual(utils.append(data, 1), [1]);
});
test('UtilsAppendUndefinedValue', () => {
const data = [1];
assert.equal(utils.append(data, undefined), data);
});
test('UtilsFindEmpty', () => {
const data: number[] = [];
assert.equal(utils.find(data, e => true), undefined);
});
test('UtilsFindNoMatch', () => {
const data = [1];
assert.equal(utils.find(data, e => false), undefined);
});
test('UtilsFindMatchSimple', () => {
const data = [1];
assert.equal(utils.find(data, e => e == 1), 1);
});
test('UtilsFindMatch', () => {
const data = [new D(1, "Hello")];
assert.equal(utils.find(data, e => e.Value == 1), data[0]);
});
test('UtilsFindMatchCovariant', () => {
const item1 = new D(1, "Hello");
const item2 = new D(2, "Hello2");
const data: B[] = [new B(0), item1, item2, new B(3)];
assert.equal(utils.find(data, (e: D) => e.Value == 2), item2);
});
test('UtilsStableSort', () => {
const data = [new D(2, "Hello3"), new D(1, "Hello1"), new D(2, "Hello4"), new D(1, "Hello2")];
const sorted = utils.stableSort(data, (a, b) => compareValues(a.Value, b.Value));
const result: string[] = [];
sorted.forEach(e => result.push(e.Name));
assert.deepEqual(result, ["Hello1", "Hello2", "Hello3", "Hello4"])
});
test('UtilsBinarySearch', () => {
const data = [new D(1, "Hello3"), new D(2, "Hello1"), new D(3, "Hello4"), new D(4, "Hello2")];
const index = utils.binarySearch(data, new D(3, "Unused"), v => v.Value, compareValues, 0);
assert.equal(index, 2);
});
test('UtilsBinarySearchMiss', () => {
const data = [new D(1, "Hello3"), new D(2, "Hello1"), new D(4, "Hello4"), new D(5, "Hello2")];
const index = utils.binarySearch(data, new D(3, "Unused"), v => v.Value, compareValues, 0);
assert.equal(~index, 2);
});
test('isArray1', () => {
const data = [new D(1, "Hello3")];
assert(isArray(data));
});
test('isArray2', () => {
const data = {};
assert(!isArray(data));
});
test('addRange1', () => {
const data: number[] = [];
assert.deepEqual(utils.addRange(data, [1, 2, 3]), [1, 2, 3]);
});
test('addRange2', () => {
const data: number[] = [1, 2, 3];
assert.deepEqual(utils.addRange(data, [1, 2, 3, 4], 3, 4), [1, 2, 3, 4]);
});
test('insertAt1', () => {
const data: number[] = [2, 3, 4];
assert.deepEqual(utils.insertAt(data, 0, 1), [1, 2, 3, 4]);
});
test('insertAt2', () => {
const data: number[] = [1, 2, 4];
assert.deepEqual(utils.insertAt(data, 2, 3), [1, 2, 3, 4]);
});
test('insertAt3', () => {
const data: number[] = [1, 2, 3];
assert.deepEqual(utils.insertAt(data, 3, 4), [1, 2, 3, 4]);
});
test('cloneAndSort', () => {
const data: number[] = [3, 2, 1];
assert.deepEqual(utils.cloneAndSort(data), [1, 2, 3]);
});
test('flatten', () => {
const data: number[][] = [[1, 2], [3, 4], [5, 6]];
assert.deepEqual(utils.flatten(data), [1, 2, 3, 4, 5, 6]);
});
class B {
Value: number;
constructor(value: number) {
this.Value = value;
}
}
class D extends B {
Name: string;
constructor(value: number, name: string) {
super(value);
this.Name = name;
}
}

60
server/src/tests/config.test.ts
View File

@ -1,11 +1,11 @@
/*
* config.test.ts
* Copyright (c) Microsoft Corporation.
* Licensed under the MIT license.
* Author: Eric Traut
*
* Unit tests for parsing of pyrightconfig.json files.
*/
* config.test.ts
* Copyright (c) Microsoft Corporation.
* Licensed under the MIT license.
* Author: Eric Traut
*
* Unit tests for parsing of pyrightconfig.json files.
*/
import * as assert from 'assert';
@ -13,20 +13,22 @@ import { AnalyzerService } from '../analyzer/service';
import { CommandLineOptions } from '../common/commandLineOptions';
import { ConfigOptions, ExecutionEnvironment } from '../common/configOptions';
import { NullConsole } from '../common/console';
import { combinePaths, normalizeSlashes } from '../common/pathUtils';
import { combinePaths, normalizeSlashes, normalizePath } from '../common/pathUtils';
import { createFromRealFileSystem } from '../common/vfs';
test('FindFilesWithConfigFile', () => {
const service = new AnalyzerService('<default>', new NullConsole());
const commandLineOptions = new CommandLineOptions(process.cwd(), true);
const cwd = normalizePath(combinePaths(process.cwd(), "../server"))
const service = new AnalyzerService('<default>', createFromRealFileSystem(), new NullConsole());
const commandLineOptions = new CommandLineOptions(cwd, true);
commandLineOptions.configFilePath = 'src/tests/samples/project1';
const configOptions = service.test_getConfigOptions(commandLineOptions);
service.setOptions(commandLineOptions);
// The config file specifies a single file spec (a directory).
assert.equal(configOptions.include.length, 1);
assert.equal(configOptions.include.length, 1, `failed creating options from ${ cwd }`);
assert.equal(normalizeSlashes(configOptions.projectRoot),
normalizeSlashes(combinePaths(process.cwd(), commandLineOptions.configFilePath)));
normalizeSlashes(combinePaths(cwd, commandLineOptions.configFilePath)));
const fileList = service.test_getFileNamesFromFileSpecs();
@ -37,9 +39,10 @@ test('FindFilesWithConfigFile', () => {
});
test('FileSpecNotAnArray', () => {
const cwd = normalizePath(combinePaths(process.cwd(), "../server"))
const nullConsole = new NullConsole();
const service = new AnalyzerService('<default>', nullConsole);
const commandLineOptions = new CommandLineOptions(process.cwd(), false);
const service = new AnalyzerService('<default>', createFromRealFileSystem(nullConsole), nullConsole);
const commandLineOptions = new CommandLineOptions(cwd, false);
commandLineOptions.configFilePath = 'src/tests/samples/project2';
service.setOptions(commandLineOptions);
@ -50,9 +53,10 @@ test('FileSpecNotAnArray', () => {
});
test('FileSpecNotAString', () => {
const cwd = normalizePath(combinePaths(process.cwd(), "../server"))
const nullConsole = new NullConsole();
const service = new AnalyzerService('<default>', nullConsole);
const commandLineOptions = new CommandLineOptions(process.cwd(), false);
const service = new AnalyzerService('<default>', createFromRealFileSystem(nullConsole), nullConsole);
const commandLineOptions = new CommandLineOptions(cwd, false);
commandLineOptions.configFilePath = 'src/tests/samples/project3';
service.setOptions(commandLineOptions);
@ -63,9 +67,10 @@ test('FileSpecNotAString', () => {
});
test('SomeFileSpecsAreInvalid', () => {
const cwd = normalizePath(combinePaths(process.cwd(), "../server"))
const nullConsole = new NullConsole();
const service = new AnalyzerService('<default>', nullConsole);
const commandLineOptions = new CommandLineOptions(process.cwd(), false);
const service = new AnalyzerService('<default>', createFromRealFileSystem(nullConsole), nullConsole);
const commandLineOptions = new CommandLineOptions(cwd, false);
commandLineOptions.configFilePath = 'src/tests/samples/project4';
service.setOptions(commandLineOptions);
@ -73,10 +78,10 @@ test('SomeFileSpecsAreInvalid', () => {
// The config file specifies four file specs in the include array
// and one in the exclude array.
assert.equal(configOptions.include.length, 4);
assert.equal(configOptions.include.length, 4, `failed creating options from ${ cwd }`);
assert.equal(configOptions.exclude.length, 1);
assert.equal(normalizeSlashes(configOptions.projectRoot),
normalizeSlashes(combinePaths(process.cwd(), commandLineOptions.configFilePath)));
normalizeSlashes(combinePaths(cwd, commandLineOptions.configFilePath)));
const fileList = service.test_getFileNamesFromFileSpecs();
@ -85,9 +90,10 @@ test('SomeFileSpecsAreInvalid', () => {
});
test('ConfigBadJson', () => {
const cwd = normalizePath(combinePaths(process.cwd(), "../server"))
const nullConsole = new NullConsole();
const service = new AnalyzerService('<default>', nullConsole);
const commandLineOptions = new CommandLineOptions(process.cwd(), false);
const service = new AnalyzerService('<default>', createFromRealFileSystem(nullConsole), nullConsole);
const commandLineOptions = new CommandLineOptions(cwd, false);
commandLineOptions.configFilePath = 'src/tests/samples/project5';
service.setOptions(commandLineOptions);
@ -98,7 +104,8 @@ test('ConfigBadJson', () => {
});
test('FindExecEnv1', () => {
const configOptions = new ConfigOptions(process.cwd());
const cwd = normalizePath(combinePaths(process.cwd(), "../server"))
const configOptions = new ConfigOptions(cwd);
// Build a config option with three execution environments.
const execEnv1 = new ExecutionEnvironment('src/foo');
@ -106,9 +113,9 @@ test('FindExecEnv1', () => {
const execEnv2 = new ExecutionEnvironment('src');
configOptions.executionEnvironments.push(execEnv2);
const file1 = normalizeSlashes(combinePaths(process.cwd(), 'src/foo/bar.py'));
const file1 = normalizeSlashes(combinePaths(cwd, 'src/foo/bar.py'));
assert.equal(configOptions.findExecEnvironment(file1), execEnv1);
const file2 = normalizeSlashes(combinePaths(process.cwd(), 'src/foo2/bar.py'));
const file2 = normalizeSlashes(combinePaths(cwd, 'src/foo2/bar.py'));
assert.equal(configOptions.findExecEnvironment(file2), execEnv2);
// If none of the execution environments matched, we should get
@ -120,8 +127,9 @@ test('FindExecEnv1', () => {
});
test('PythonPlatform', () => {
const cwd = normalizePath(combinePaths(process.cwd(), "../server"))
const nullConsole = new NullConsole();
const configOptions = new ConfigOptions(process.cwd());
const configOptions = new ConfigOptions(cwd);
const json = JSON.parse(`{
"executionEnvironments" : [

97
server/src/tests/debug.test.ts Normal file
View File

@ -0,0 +1,97 @@
/*
* debug.test.ts
* Copyright (c) Microsoft Corporation.
* Licensed under the MIT license.
*/
import * as assert from 'assert';
import * as debug from '../common/debug';
test('DebugAssertTrue', () => {
assert.doesNotThrow(() => debug.assert(true, "doesn't throw"));
});
test('DebugAssertFalse', () => {
assert.throws(
() => debug.assert(false, "should throw"),
(err: any) => err instanceof Error,
"unexpected");
});
test('DebugAssertDetailInfo', () => {
// let assert to show more detail info which will get collected when
// assert raised
const detailInfo = "Detail Info";
assert.throws(
() => debug.assert(false, "should throw", () => detailInfo),
(err: any) => err instanceof Error && err.message.includes(detailInfo),
"unexpected");
});
test('DebugAssertStackTrace', () => {
// let assert to control what callstack to put in exception stack
assert.throws(
() => debug.assert(false, "should throw", undefined, assert.throws),
(err: any) => err instanceof Error && !err.message.includes("assert.throws"),
"unexpected");
});
test('DebugAssertUndefined', () => {
const unused = undefined;
assert.throws(
() => debug.assertDefined(unused),
(err: any) => err instanceof Error,
"unexpected");
});
test('DebugAssertDefined', () => {
const unused = 1;
assert.doesNotThrow(() => debug.assertDefined(unused));
});
test('DebugAssertEachUndefined', () => {
type T = number | undefined;
const unused: T[] = [1, 2, 3, undefined];
assert.throws(
() => debug.assertEachDefined(unused),
(err: any) => err instanceof Error,
"unexpected");
});
test('DebugAssertEachDefined', () => {
const unused: number[] = [1, 2, 3];
assert.doesNotThrow(() => debug.assertEachDefined(unused));
});
test('DebugAssertNever', () => {
const enum MyEnum { A, B, C }
const unused = 5 as MyEnum;
// prevent one from adding new values and forget to add
// handlers some places
assert.throws(
() => {
switch (unused) {
case MyEnum.A:
case MyEnum.B:
case MyEnum.C:
break;
default:
debug.assertNever(unused);
}
},
(err: any) => err instanceof Error,
"unexpected");
});
test('DebugGetFunctionName', () => {
// helper method to add better message in exception
assert(debug.getFunctionName(assert.throws) === "throws");
});
test('DebugFormatEnum', () => {
// helper method to add better message in exception around enum
// const enum require --preserveConstEnums flag to work properly
enum MyEnum { A, B, C }
assert(debug.formatEnum(MyEnum.A, MyEnum, false) === "A");
});

205
server/src/tests/filesystem.test.ts Normal file
View File

@ -0,0 +1,205 @@
/*
* filesystem.test.ts
* Copyright (c) Microsoft Corporation.
* Licensed under the MIT license.
*
* Test and Show how to use virtual file system
*/
import * as assert from 'assert';
import * as host from './harness/host';
import * as vfs from "./harness/vfs/filesystem";
import * as factory from "./harness/vfs/factory"
import { normalizeSlashes, combinePaths } from '../common/pathUtils';
test('CreateVFS', () => {
const cwd = normalizeSlashes("/");
const fs = new vfs.FileSystem(/*ignoreCase*/ true, { cwd: cwd })
assert.equal(fs.cwd(), cwd)
});
test('Folders', () => {
const cwd = normalizeSlashes("/");
const fs = new vfs.FileSystem(/*ignoreCase*/ true, { cwd: cwd })
// no such dir exist
assert.throws(() => fs.chdir("a"))
fs.mkdirSync("a");
fs.chdir("a");
assert.equal(fs.cwd(), normalizeSlashes("/a"));
fs.chdir("..");
fs.rmdirSync("a");
// no such dir exist
assert.throws(() => fs.chdir("a"))
});
test('Files', () => {
const cwd = normalizeSlashes("/");
const fs = new vfs.FileSystem(/*ignoreCase*/ true, { cwd: cwd })
fs.writeFileSync("1.txt", "hello", "utf8");
const buffer1 = fs.readFileSync("1.txt");
assert.equal(buffer1.toString(), "hello");
const p = normalizeSlashes("a/b/c");
fs.mkdirpSync(p);
const f = combinePaths(p, "2.txt");
fs.writeFileSync(f, "hi");
const str = fs.readFileSync(f, "utf8");
assert.equal(str, "hi");
});
test('CreateRich', () => {
const cwd = normalizeSlashes("/");
const files: vfs.FileSet = {
[normalizeSlashes("/a/b/c/1.txt")]: new vfs.File("hello1"),
[normalizeSlashes("/a/b/2.txt")]: new vfs.File("hello2"),
[normalizeSlashes("/a/3.txt")]: new vfs.File("hello3"),
[normalizeSlashes("/4.txt")]: new vfs.File("hello4", { encoding: "utf16le" }),
[normalizeSlashes("/a/b/../c/./5.txt")]: new vfs.File("hello5", { encoding: "ucs2" })
}
const fs = new vfs.FileSystem(/*ignoreCase*/ true, { cwd: cwd, files: files })
const entries = fs.scanSync(cwd, "descendants-or-self", {});
// files + directory + root
assert.equal(entries.length, 10);
assert.equal(fs.readFileSync(normalizeSlashes("/a/b/c/1.txt"), "ascii"), "hello1");
assert.equal(fs.readFileSync(normalizeSlashes("/a/b/2.txt"), "utf8"), "hello2");
assert.equal(fs.readFileSync(normalizeSlashes("/a/3.txt"), "utf-8"), "hello3");
assert.equal(fs.readFileSync(normalizeSlashes("/4.txt"), "utf16le"), "hello4");
assert.equal(fs.readFileSync(normalizeSlashes("/a/c/5.txt"), "ucs2"), "hello5");
});
test('Shadow', () => {
const cwd = normalizeSlashes("/");
const fs = new vfs.FileSystem(/*ignoreCase*/ true, { cwd: cwd });
// only readonly fs can be shadowed
assert.throws(() => fs.shadow());
// one way to create shadow is making itself snapshot
fs.snapshot();
assert(!fs.isReadonly);
assert(fs.shadowRoot!.isReadonly);
// another way is creating one off existing readonly snapshot
const shadow1 = fs.shadowRoot!.shadow();
assert(!shadow1.isReadonly);
assert(shadow1.shadowRoot === fs.shadowRoot);
// make itself readonly and then shawdow
shadow1.makeReadonly();
assert(shadow1.isReadonly);
const shadow2 = shadow1.shadow();
assert(!shadow2.isReadonly);
assert(shadow2.shadowRoot === shadow1);
});
test('Diffing', () => {
const cwd = normalizeSlashes("/");
const fs = new vfs.FileSystem(/*ignoreCase*/ true, { cwd: cwd });
// first snapshot
fs.snapshot();
fs.writeFileSync("test1.txt", "hello1");
// compared with original
assert.equal(countFile(fs.diff()!), 1);
// second snapshot
fs.snapshot();
fs.writeFileSync("test2.txt", "hello2");
// compared with first snapshot
assert.equal(countFile(fs.diff()!), 1);
// compare with original snapshot
assert.equal(countFile(fs.diff(fs.shadowRoot!.shadowRoot)!), 2);
// branch out from first snapshot
const s = fs.shadowRoot!.shadow();
// "test2.txt" only exist in first snapshot
assert(!s.existsSync("test2.txt"));
// create parallel universe where it has another version of test2.txt with different content
// compared to second snapshot which forked from same first snapshot
s.writeFileSync("test2.txt", "hello3");
// diff between non direct snapshots
// diff gives test2.txt even though it exist in both snapshot
assert.equal(countFile(s.diff(fs)!), 1);
});
test('createFromFileSystem1', () => {
const filepath = normalizeSlashes(combinePaths(factory.srcFolder, "test.py"));
const content = "# test";
// file system will map physical file system to virtual one
const fs = factory.createFromFileSystem(host.Host, false, { documents: [new factory.TextDocument(filepath, content)], cwd: factory.srcFolder });
// check existing typeshed folder on virtual path inherited from base snapshot from physical file system
const entries = fs.readdirSync(factory.typeshedFolder);
assert(entries.length > 0);
// confirm file
assert.equal(fs.readFileSync(filepath, "utf8"), content);
});
test('createFromFileSystem2', () => {
const fs = factory.createFromFileSystem(host.Host, /* ignoreCase */ true, { cwd: factory.srcFolder });
const entries = fs.readdirSync(factory.typeshedFolder.toUpperCase());
assert(entries.length > 0);
});
test('createFromFileSystemWithCustomTypeshedPath', () => {
const invalidpath = normalizeSlashes(combinePaths(host.Host.getWorkspaceRoot(), "../docs"));
const fs = factory.createFromFileSystem(host.Host, /* ignoreCase */ false, {
cwd: factory.srcFolder, meta: { [factory.typeshedFolder]: invalidpath }
});
const entries = fs.readdirSync(factory.typeshedFolder);
assert(entries.filter(e => e.endsWith(".md")).length > 0);
});
test('createFromFileSystemWithMetadata', () => {
const fs = factory.createFromFileSystem(host.Host, /* ignoreCase */ false, {
cwd: factory.srcFolder, meta: { "unused": "unused" }
});
assert(fs.existsSync(factory.srcFolder));
});
function countFile(files: vfs.FileSet): number {
let count = 0;
for (const value of Object.values(flatten(files))) {
if (value instanceof vfs.File) {
count++;
}
}
return count;
}
function flatten(files: vfs.FileSet): vfs.FileSet {
const result: vfs.FileSet = {};
_flatten(files, result);
return result;
}
function _flatten(files: vfs.FileSet, result: vfs.FileSet): void {
for (const [key, value] of Object.entries(files)) {
result[key] = value;
if (value instanceof vfs.Directory) {
_flatten(value.files, result);
}
}
}

291
server/src/tests/fourSlashParser.test.ts Normal file
View File

@ -0,0 +1,291 @@
/*
* fourSlashParser.test.ts
* Copyright (c) Microsoft Corporation.
* Licensed under the MIT license.
*
* Tests and show how to use fourslash markup languages
* and how to use parseTestData API itself for other unit tests
*/
import * as assert from 'assert';
import * as factory from "./harness/vfs/factory"
import * as host from './harness/host';
import { parseTestData } from './harness/fourslash/fourSlashParser';
import { compareStringsCaseSensitive } from '../common/stringUtils';
import { CompilerSettings } from './harness/fourslash/fourSlashTypes';
import { normalizeSlashes, getBaseFileName } from '../common/pathUtils';
test('GlobalOptions', () => {
const code = `
// global options
// @libpath: ../dist/lib
// @pythonversion: 3.7
////class A:
//// pass
`
const content = `class A:
pass`;
const data = parseTestData(".", code, "test.py");
assertOptions(data.globalOptions, [["libpath", "../dist/lib"], ["pythonversion", "3.7"]]);
assert.equal(data.files.length, 1);
assert.equal(data.files[0].fileName, "test.py");
assert.equal(data.files[0].content, content);
});
test('Filename', () => {
const code = `
// @filename: file1.py
////class A:
//// pass
`
const content = `class A:
pass`;
const data = parseTestData(".", code, "test.py");
assertOptions(data.globalOptions, []);
assert.equal(data.files.length, 1);
assert.equal(data.files[0].fileName, normalizeSlashes("./file1.py"));
assert.equal(data.files[0].content, content);
});
test('Extra file options', () => {
// filename must be last file options
const code = `
// @reserved: not used
// @filename: file1.py
////class A:
//// pass
`
const data = parseTestData(".", code, "test.py");
assertOptions(data.globalOptions, []);
assertOptions(data.files[0].fileOptions, [["filename", "file1.py"], ["reserved", "not used"]])
});
test('Range', () => {
const code = `
////class A:
//// [|pass|]
`
const content = `class A:
pass`;
const data = parseTestData(".", code, "test.py");
assert.equal(data.files[0].content, content);
assert.deepEqual(data.ranges, [{ fileName: "test.py", pos: 13, end: 17, marker: undefined }]);
});
test('Marker', () => {
const code = `
////class A:
//// /*marker1*/pass
`
const content = `class A:
pass`;
const data = parseTestData(".", code, "test.py");
assert.equal(data.files[0].content, content);
const marker = { fileName: "test.py", position: 13 };
assert.deepEqual(data.markers, [marker]);
assert.deepEqual(data.markerPositions.get("marker1"), marker)
});
test('MarkerWithData', () => {
// embeded json data
const code = `
////class A:
//// {| "data1":"1", "data2":"2" |}pass
`
const content = `class A:
pass`;
const data = parseTestData(".", code, "test.py");
assert.equal(data.files[0].content, content);
assert.deepEqual(data.markers, [{ fileName: "test.py", position: 13, data: { data1: "1", data2: "2" } }]);
assert.equal(data.markerPositions.size, 0)
});
test('MarkerWithDataAndName', () => {
// embeded json data with "name"
const code = `
////class A:
//// {| "name": "marker1", "data1":"1", "data2":"2" |}pass
`
const content = `class A:
pass`;
const data = parseTestData(".", code, "test.py");
assert.equal(data.files[0].content, content);
const marker = { fileName: "test.py", position: 13, data: { name: "marker1", data1: "1", data2: "2" } };
assert.deepEqual(data.markers, [marker]);
assert.deepEqual(data.markerPositions.get(marker.data.name), marker)
});
test('RangeWithMarker', () => {
// range can have 1 marker in it
const code = `
////class A:
//// [|/*marker1*/pass|]
`
const content = `class A:
pass`;
const data = parseTestData(".", code, "test.py");
assert.equal(data.files[0].content, content);
const marker = { fileName: "test.py", position: 13 };
assert.deepEqual(data.markers, [marker]);
assert.deepEqual(data.markerPositions.get("marker1"), marker)
assert.deepEqual(data.ranges, [{ fileName: "test.py", pos: 13, end: 17, marker: marker }]);
});
test('RangeWithMarkerAndJsonData', () => {
// range can have 1 marker in it
const code = `
////class A:
//// [|{| "name": "marker1", "data1":"1", "data2":"2" |}pass|]
`
const content = `class A:
pass`;
const data = parseTestData(".", code, "test.py");
assert.equal(data.files[0].content, content);
const marker = { fileName: "test.py", position: 13, data: { name: "marker1", data1: "1", data2: "2" } };
assert.deepEqual(data.markers, [marker]);
assert.deepEqual(data.markerPositions.get(marker.data.name), marker)
assert.deepEqual(data.ranges, [{ fileName: "test.py", pos: 13, end: 17, marker: marker }]);
});
test('Multiple Files', () => {
// range can have 1 marker in it
const code = `
// @filename: src/A.py
////class A:
//// pass
// @filename: src/B.py
////class B:
//// pass
// @filename: src/C.py
////class C:
//// pass
`
const data = parseTestData(".", code, "test.py");
assert.equal(data.files.length, 3);
assert.equal(data.files.filter(f => f.fileName === normalizeSlashes("./src/A.py"))[0].content, getContent("A"));
assert.equal(data.files.filter(f => f.fileName === normalizeSlashes("./src/B.py"))[0].content, getContent("B"));
assert.equal(data.files.filter(f => f.fileName === normalizeSlashes("./src/C.py"))[0].content, getContent("C"));
});
test('Multiple Files with default name', () => {
// only very first one can omit filename
const code = `
////class A:
//// pass
// @filename: src/B.py
////class B:
//// pass
// @filename: src/C.py
////class C:
//// pass
`
const data = parseTestData(".", code, "./src/test.py");
assert.equal(data.files.length, 3);
assert.equal(data.files.filter(f => f.fileName === normalizeSlashes("./src/test.py"))[0].content, getContent("A"));
assert.equal(data.files.filter(f => f.fileName === normalizeSlashes("./src/B.py"))[0].content, getContent("B"));
assert.equal(data.files.filter(f => f.fileName === normalizeSlashes("./src/C.py"))[0].content, getContent("C"));
});
test('Multiple Files with markers', () => {
// range can have 1 marker in it
const code = `
// @filename: src/A.py
////class A:
//// [|pass|]
// @filename: src/B.py
////class B:
//// [|/*marker1*/pass|]
// @filename: src/C.py
////class C:
//// [|{|"name":"marker2", "data":"2"|}pass|]
`
const data = parseTestData(".", code, "test.py");
assert.equal(data.files.length, 3);
assert.equal(data.files.filter(f => f.fileName === normalizeSlashes("./src/A.py"))[0].content, getContent("A"));
assert.equal(data.files.filter(f => f.fileName === normalizeSlashes("./src/B.py"))[0].content, getContent("B"));
assert.equal(data.files.filter(f => f.fileName === normalizeSlashes("./src/C.py"))[0].content, getContent("C"));
assert.equal(data.ranges.length, 3);
assert(data.markerPositions.get("marker1"));
assert(data.markerPositions.get("marker2"));
assert.equal(data.ranges.filter(r => r.marker).length, 2);
});
test('fourSlashWithFileSystem', () => {
const code = `
// @filename: src/A.py
////class A:
//// pass
// @filename: src/B.py
////class B:
//// pass
// @filename: src/C.py
////class C:
//// pass
`
const data = parseTestData(".", code, "unused");
const documents = data.files.map(f => new factory.TextDocument(f.fileName, f.content, new Map<string, string>(Object.entries(f.fileOptions))));
const fs = factory.createFromFileSystem(host.Host, /* ignoreCase */ false, { documents: documents, cwd: normalizeSlashes("/") });
for (const file of data.files) {
assert.equal(fs.readFileSync(file.fileName, "utf8"), getContent(getBaseFileName(file.fileName, ".py", false)));
}
});
function getContent(className: string) {
return `class ${className}:
pass`;
}
function assertOptions(actual: CompilerSettings, expected: [string, string][], message?: string | Error): void {
assert.deepEqual(
Object.entries(actual).sort((x, y) => compareStringsCaseSensitive(x[0], y[0])),
expected,
message);
}

33
server/src/tests/fourSlashRunner.test.ts Normal file
View File

@ -0,0 +1,33 @@
/*
* fourslashrunner.test.ts
* Copyright (c) Microsoft Corporation.
* Licensed under the MIT license.
*
* Entry point that will read all *.fourslash.ts files and
* register jest tests for them and run
*/
import * as path from "path";
import * as host from "./harness/host";
import { normalizeSlashes } from "../common/pathUtils";
import { runFourSlashTest } from "./harness/fourslash/runner";
import { srcFolder } from "./harness/vfs/factory";
describe("fourslash tests", () => {
const testFiles: string[] = [];
const basePath = path.resolve(path.dirname(module.filename), "fourslash/");
for (const file of host.Host.listFiles(basePath, /.*\.fourslash\.ts$/i, { recursive: true })) {
testFiles.push(file);
}
testFiles.forEach(file => {
describe(file, () => {
const fn = normalizeSlashes(file);
const justName = fn.replace(/^.*[\\/]/, "");
it("fourslash test " + justName + " runs correctly", () => {
runFourSlashTest(srcFolder, fn);
});
});
});
});

33
server/src/tests/fourslash/dataclass1.fourslash.ts Normal file
View File

@ -0,0 +1,33 @@
/// <reference path="fourslash.ts" />
// @filename: dataclass1.py
//// # This sample validates the Python 3.7 data class feature.
////
//// from typing import NamedTuple, Optional
////
//// class Other:
//// pass
////
//// class DataTuple(NamedTuple):
//// def _m(self):
//// pass
//// id: int
//// aid: Other
//// valll: str = ''
//// name: Optional[str] = None
////
//// d1 = DataTuple(id=1, aid=Other())
//// d2 = DataTuple(id=1, aid=Other(), valll='v')
//// d3 = DataTuple(id=1, aid=Other(), name='hello')
//// d4 = DataTuple(id=1, aid=Other(), name=None)
//// id = d1.id
////
//// # This should generate an error because the name argument
//// # is the incorrect type.
//// d5 = DataTuple(id=1, aid=Other(), name=[|{|"category": "error"|}3|])
////
//// # This should generate an error because aid is a required
//// # parameter and is missing an argument here.
//// d6 = [|{|"category": "error"|}DataTuple(id=1, name=None|])
helper.verifyDiagnostics();

20
server/src/tests/fourslash/dataclass3.fourslash.ts Normal file
View File

@ -0,0 +1,20 @@
/// <reference path="fourslash.ts" />
// @filename: dataclass3.py
//// # This sample validates the Python 3.7 data class feature, ensuring that
//// # NamedTuple must be a direct base class.
////
//// from typing import NamedTuple
////
//// class Parent(NamedTuple):
//// pass
////
//// class DataTuple2(Parent):
//// id: int
////
//// # This should generate an error because DataTuple2 isn't considered
//// # a data class and won't have the associated __new__ or __init__
//// # method defined.
//// data = DataTuple2([|{|"category": "error"|}id|]=1)
helper.verifyDiagnostics();

58
server/src/tests/fourslash/dataclass4.fourslash.ts Normal file
View File

@ -0,0 +1,58 @@
/// <reference path="fourslash.ts" />
//// # This sample tests the handling of the @dataclass decorator.
////
//// from dataclasses import dataclass, InitVar
////
//// @dataclass
//// class Bar():
//// bbb: int
//// ccc: str
//// aaa = 'string'
////
//// bar1 = Bar(bbb=5, ccc='hello')
//// bar2 = Bar(5, 'hello')
//// bar3 = Bar(5, 'hello', 'hello2')
//// print(bar3.bbb)
//// print(bar3.ccc)
//// print(bar3.aaa)
////
//// # This should generate an error because ddd
//// # isn't a declared value.
//// bar = Bar(bbb=5, [|/*marker1*/ddd|]=5, ccc='hello')
////
//// # This should generate an error because the
//// # parameter types don't match.
//// bar = Bar([|/*marker2*/'hello'|], 'goodbye')
////
//// # This should generate an error because a parameter
//// # is missing.
//// bar = [|/*marker3*/Bar(2)|]
////
//// # This should generate an error because there are
//// # too many parameters.
//// bar = Bar(2, 'hello', 'hello', [|/*marker4*/4|])
////
////
//// @dataclass
//// class Baz1():
//// bbb: int
//// aaa = 'string'
////
//// # This should generate an error because variables
//// # with no default cannot come after those with
//// # defaults.
//// [|/*marker5*/ccc|]: str
////
//// @dataclass
//// class Baz2():
//// aaa: str
//// ddd: InitVar[int] = 3
helper.verifyDiagnostics({
"marker1": { category: "error", message: "No parameter named 'ddd'" },
"marker2": { category: "error", message: "Argument of type 'Literal['hello']' cannot be assigned to parameter 'bbb' of type 'int'\n 'str' is incompatible with 'int'" },
"marker3": { category: "error", message: "Argument missing for parameter 'ccc'" },
"marker4": { category: "error", message: "Expected 3 positional arguments" },
"marker5": { category: "error", message: "Data fields without default value cannot appear after data fields with default values" },
});

53
server/src/tests/fourslash/dataclass5.fourslash.ts Normal file
View File

@ -0,0 +1,53 @@
/// <reference path="fourslash.ts" />
//// # This sample tests the handling of the @dataclass decorator
//// # with a custom __init__.
////
//// from dataclasses import dataclass
////
//// @dataclass(init=False)
//// class A:
//// x: int
//// x_squared: int
////
//// def __init__(self, x: int):
//// self.x = x
//// self.x_squared = x ** 2
////
//// a = A(3)
////
//// @dataclass(init=True)
//// class B:
//// x: int
//// x_squared: int
////
//// def __init__(self, x: int):
//// self.x = x
//// self.x_squared = x ** 2
////
//// b = B(3)
////
//// @dataclass()
//// class C:
//// x: int
//// x_squared: int
////
//// def __init__(self, x: int):
//// self.x = x
//// self.x_squared = x ** 2
////
//// c = C(3)
////
//// @dataclass(init=False)
//// class D:
//// x: int
//// x_squared: int
////
//// # This should generate an error because there is no
//// # override __init__ method and no synthesized __init__.
//// d = [|/*marker1*/[|/*marker2*/D(3|]|])
helper.verifyDiagnostics({
"marker1": { category: "error", message: "Expected no arguments to 'D' constructor" },
"marker2": { category: "error", message: "'D(3)' has type 'Type[D]' and is not callable" },
});

34
server/src/tests/fourslash/dataclass6.fourslash.ts Normal file
View File

@ -0,0 +1,34 @@
/// <reference path="fourslash.ts" />
//// # This sample tests the type checker's handling of
//// # synthesized __init__ and __new__ methods for
//// # dataclass classes and their subclasses.
////
//// from dataclasses import dataclass
////
//// @dataclass
//// class A:
//// x: int
////
//// @dataclass(init)
//// class B(A):
//// y: int
////
//// def __init__(self, a: A, y: int):
//// self.__dict__ = a.__dict__
////
//// a = A(3)
//// b = B(a, 5)
////
////
//// # This should generate an error because there is an extra parameter
//// a = A(3, [|/*marker1*/4|])
////
//// # This should generate an error because there is one too few parameters
//// b = [|/*marker2*/B(a)|]
////
helper.verifyDiagnostics({
"marker1": { category: "error", message: "Expected 1 positional argument" },
"marker2": { category: "error", message: "Argument missing for parameter 'y'" },
});

59
server/src/tests/fourslash/dataclass7.fourslash.ts Normal file
View File

@ -0,0 +1,59 @@
/// <reference path="fourslash.ts" />
//// # This sample tests the analyzer's ability to handline inherited
//// # data classes.
////
//// from dataclasses import dataclass
////
//// class C1: ...
//// class C2: ...
//// class C3: ...
////
//// @dataclass
//// class DC1:
//// aa: C1
//// bb: C2
//// cc: C3
////
//// class NonDC2:
//// ff: int
////
//// @dataclass
//// class DC2(NonDC2, DC1):
//// ee: C2
//// aa: C2
//// dd: C2
////
//// dc2_1 = DC2(C2(), C2(), C3(), C2(), C2())
////
//// # This should generate an error because the type
//// # of parameter aa has been replaced with type C1.
//// dc2_2 = DC2([|/*marker1*/C1()|], C2(), C3(), C2(), C2())
////
//// dc2_3 = DC2(ee=C2(), dd=C2(), aa=C2(), bb=C2(), cc=C3())
////
////
//// @dataclass
//// class DC3:
//// aa: C1
//// bb: C2 = C2()
//// cc: C3 = C3()
////
//// @dataclass
//// class DC4(DC3):
//// # This should generate an error because
//// # previous parameters have default values.
//// [|/*marker2*/dd|]: C1
////
//// @dataclass
//// class DC5(DC3):
//// # This should not generate an error because
//// # aa replaces aa in DC3, and it's ordered
//// # before the params with default values.
//// aa: C2
////
helper.verifyDiagnostics({
"marker1": { category: "error", message: "Argument of type 'C1' cannot be assigned to parameter 'aa' of type 'C2'\n 'C1' is incompatible with 'C2'" },
"marker2": { category: "error", message: "Data fields without default value cannot appear after data fields with default values" },
});

102
server/src/tests/fourslash/fourslash.ts Normal file
View File

@ -0,0 +1,102 @@
/*
* fourslash.ts
* Copyright (c) Microsoft Corporation.
* Licensed under the MIT license.
*
* this file only exists for the richer editing experiences on *.fourslash.ts files.
* when fourslash tests are actually running this file is not used.
*
* this basically provides type information through // <reference .. > while editing but
* get ignored when test run due to how test code is injected when running.
* see - server\pyright\server\src\tests\harness\fourslash\runner.ts@runCode - for more detail
*
* when run, helper variable will be bount to TestState (server\pyright\server\src\tests\harness\fourslash\testState.ts)
* so make sure Foruslash type is in sync with TestState
*
* for how markup language and helper is used in fourslash tests, see these 2 tests
* server\pyright\server\src\tests\fourSlashParser.test.ts
* server\pyright\server\src\tests\testState.test.ts
*
* for debugging, open *.fourslash.ts test file you want to debug, and select "fourslash current file" as debug configuration
* and set break point in one of TestState methods you are using in the test or set break point on "runCode" above
* and hit F5.
*/
declare namespace _ {
interface TextRange {
start: number;
length: number;
}
interface LineAndColumn {
// Both line and column are zero-based
line: number;
column: number;
}
interface Marker {
fileName: string;
position: number;
data?: {};
}
interface Range {
fileName: string;
marker?: Marker;
pos: number;
end: number;
}
interface TextChange {
span: TextRange;
newText: string;
}
interface Fourslash {
getMarkerName(m: Marker): string;
getMarkerByName(markerName: string): Marker;
getMarkerNames(): string[];
getMarkers(): Marker[];
getRanges(): Range[];
getRangesInFile(fileName: string): Range[];
getRangesByText(): Map<string, Range[]>;
goToBOF(): void;
goToEOF(): void;
goToPosition(positionOrLineAndColumn: number | LineAndColumn): void;
goToMarker(nameOrMarker: string | Marker): void;
goToEachMarker(markers: readonly Marker[], action: (marker: Marker, index: number) => void): void;
goToEachRange(action: (range: Range) => void): void;
goToRangeStart({ fileName, pos }: Range): void;
select(startMarker: string, endMarker: string): void;
selectAllInFile(fileName: string): void;
selectRange(range: Range): void;
selectLine(index: number): void;
moveCaretRight(count: number): void;
verifyDiagnostics(map?: { [marker: string]: { category: string; message: string } }): void;
/* not tested yet
openFile(indexOrName: number | string, content?: string): void;
paste(text: string): void;
type(text: string): void;
replace(start: number, length: number, text: string): void;
deleteChar(count: number): void;
deleteLineRange(startIndex: number, endIndexInclusive: number): void;
deleteCharBehindMarker(count: number): void;
verifyCaretAtMarker(markerName: string): void;
verifyCurrentLineContent(text: string): void;
verifyCurrentFileContent(text: string): void;
verifyTextAtCaretIs(text: string): void;
verifyRangeIs(expectedText: string, includeWhiteSpace?: boolean): void;
setCancelled(numberOfCalls: number): void;
resetCancelled(): void; */
}
}
declare var helper: _.Fourslash;

398
server/src/tests/harness/fourslash/fourSlashParser.ts Normal file
View File

@ -0,0 +1,398 @@
/*
* fourSlashParser.ts
* Copyright (c) Microsoft Corporation.
* Licensed under the MIT license.
*
* Parse fourslash markup code and return parsed content with marker/range data
*/
import { contains } from "../../../common/collectionUtils";
import { combinePaths, isRootedDiskPath, normalizeSlashes } from "../../../common/pathUtils";
import { fileMetadataNames, FourSlashData, FourSlashFile, Marker, MetadataOptionNames, Range } from "./fourSlashTypes";
/**
* Parse given fourslash markup code and return content with markup/range data
*
* @param basePath this will be combined with given `fileName` to form filepath to this content
* @param contents content with fourslash markups.
* @param fileName this will be a default filename for the first no named content in `contents`.
* if content is marked with `@filename`, that will override this given `filename`
*/
export function parseTestData(basePath: string, contents: string, fileName: string): FourSlashData {
const normalizedBasePath = normalizeSlashes(basePath);
// Regex for parsing options in the format "@Alpha: Value of any sort"
const optionRegex = /^\s*@(\w+):\s*(.*)\s*/;
// List of all the subfiles we've parsed out
const files: FourSlashFile[] = [];
// Global options
const globalOptions: { [s: string]: string } = {};
// Marker positions
// Split up the input file by line
// Note: IE JS engine incorrectly handles consecutive delimiters here when using RegExp split, so
// we have to string-based splitting instead and try to figure out the delimiting chars
const lines = contents.split("\n");
let i = 0;
const markerPositions = new Map<string, Marker>();
const markers: Marker[] = [];
const ranges: Range[] = [];
// Stuff related to the subfile we're parsing
let currentFileContent: string | undefined;
let currentFileName = normalizeSlashes(fileName);
let currentFileOptions: { [s: string]: string } = {};
function nextFile() {
if (currentFileContent === undefined) return;
const file = parseFileContent(currentFileContent, currentFileName, markerPositions, markers, ranges);
file.fileOptions = currentFileOptions;
// Store result file
files.push(file);
currentFileContent = undefined;
currentFileOptions = {};
currentFileName = fileName;
}
for (let line of lines) {
i++;
if (line.length > 0 && line.charAt(line.length - 1) === "\r") {
line = line.substr(0, line.length - 1);
}
if (line.substr(0, 4) === "////") {
const text = line.substr(4);
currentFileContent = currentFileContent === undefined ? text : currentFileContent + "\n" + text;
}
else if (line.substr(0, 3) === "///" && currentFileContent !== undefined) {
throw new Error("Three-slash line in the middle of four-slash region at line " + i);
}
else if (line.substr(0, 2) === "//") {
// Comment line, check for global/file @options and record them
const match = optionRegex.exec(line.substr(2));
if (match) {
const key = match[1].toLowerCase();
const value = match[2];
if (!contains(fileMetadataNames, key)) {
// Check if the match is already existed in the global options
if (globalOptions[key] !== undefined) {
throw new Error(`Global option '${ key }' already exists`);
}
globalOptions[key] = value;
}
else {
switch (key) {
case MetadataOptionNames.fileName:
{
// Found an @FileName directive, if this is not the first then create a new subfile
nextFile();
const normalizedPath = normalizeSlashes(value);
currentFileName = isRootedDiskPath(normalizedPath) ? normalizedPath : combinePaths(normalizedBasePath, normalizedPath);
currentFileOptions[key] = value;
break;
}
default:
// Add other fileMetadata flag
currentFileOptions[key] = value;
}
}
}
}
// Previously blank lines between fourslash content caused it to be considered as 2 files,
// Remove this behavior since it just causes errors now
else if (line !== "") {
// Code line, terminate current subfile if there is one
nextFile();
}
}
return {
markerPositions,
markers,
globalOptions,
files,
ranges
};
}
interface LocationInformation {
position: number;
sourcePosition: number;
sourceLine: number;
sourceColumn: number;
}
interface RangeLocationInformation extends LocationInformation {
marker?: Marker;
}
const enum State {
none,
inSlashStarMarker,
inObjectMarker
}
function reportError(fileName: string, line: number, col: number, message: string) {
const errorMessage = fileName + "(" + line + "," + col + "): " + message;
throw new Error(errorMessage);
}
function recordObjectMarker(fileName: string, location: LocationInformation, text: string, markerMap: Map<string, Marker>, markers: Marker[]): Marker | undefined {
let markerValue: any;
try {
// Attempt to parse the marker value as JSON
markerValue = JSON.parse("{ " + text + " }");
}
catch (e) {
reportError(fileName, location.sourceLine, location.sourceColumn, "Unable to parse marker text " + e.message);
}
if (markerValue === undefined) {
reportError(fileName, location.sourceLine, location.sourceColumn, "Object markers can not be empty");
return undefined;
}
const marker: Marker = {
fileName,
position: location.position,
data: markerValue
};
// Object markers can be anonymous
if (markerValue.name) {
markerMap.set(markerValue.name, marker);
}
markers.push(marker);
return marker;
}
function recordMarker(fileName: string, location: LocationInformation, name: string, markerMap: Map<string, Marker>, markers: Marker[]): Marker | undefined {
const marker: Marker = {
fileName,
position: location.position
};
// Verify markers for uniqueness
if (markerMap.has(name)) {
const message = "Marker '" + name + "' is duplicated in the source file contents.";
reportError(marker.fileName, location.sourceLine, location.sourceColumn, message);
return undefined;
}
else {
markerMap.set(name, marker);
markers.push(marker);
return marker;
}
}
function parseFileContent(content: string, fileName: string, markerMap: Map<string, Marker>, markers: Marker[], ranges: Range[]): FourSlashFile {
content = chompLeadingSpace(content);
// Any slash-star comment with a character not in this string is not a marker.
const validMarkerChars = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz$1234567890_";
/// The file content (minus metacharacters) so far
let output = "";
/// The current marker (or maybe multi-line comment?) we're parsing, possibly
let openMarker: LocationInformation | undefined;
/// A stack of the open range markers that are still unclosed
const openRanges: RangeLocationInformation[] = [];
/// A list of ranges we've collected so far */
let localRanges: Range[] = [];
/// The latest position of the start of an unflushed plain text area
let lastNormalCharPosition = 0;
/// The total number of metacharacters removed from the file (so far)
let difference = 0;
/// The fourslash file state object we are generating
let state: State = State.none;
/// Current position data
let line = 1;
let column = 1;
const flush = (lastSafeCharIndex: number | undefined) => {
output = output + content.substr(lastNormalCharPosition, lastSafeCharIndex === undefined ? undefined : lastSafeCharIndex - lastNormalCharPosition);
};
if (content.length > 0) {
let previousChar = content.charAt(0);
for (let i = 1; i < content.length; i++) {
const currentChar = content.charAt(i);
switch (state) {
case State.none:
if (previousChar === "[" && currentChar === "|") {
// found a range start
openRanges.push({
position: (i - 1) - difference,
sourcePosition: i - 1,
sourceLine: line,
sourceColumn: column,
});
// copy all text up to marker position
flush(i - 1);
lastNormalCharPosition = i + 1;
difference += 2;
}
else if (previousChar === "|" && currentChar === "]") {
// found a range end
const rangeStart = openRanges.pop();
if (!rangeStart) {
throw reportError(fileName, line, column, "Found range end with no matching start.");
}
const range: Range = {
fileName,
pos: rangeStart.position,
end: (i - 1) - difference,
marker: rangeStart.marker
};
localRanges.push(range);
// copy all text up to range marker position
flush(i - 1);
lastNormalCharPosition = i + 1;
difference += 2;
}
else if (previousChar === "/" && currentChar === "*") {
// found a possible marker start
state = State.inSlashStarMarker;
openMarker = {
position: (i - 1) - difference,
sourcePosition: i - 1,
sourceLine: line,
sourceColumn: column,
};
}
else if (previousChar === "{" && currentChar === "|") {
// found an object marker start
state = State.inObjectMarker;
openMarker = {
position: (i - 1) - difference,
sourcePosition: i - 1,
sourceLine: line,
sourceColumn: column,
};
flush(i - 1);
}
break;
case State.inObjectMarker:
// Object markers are only ever terminated by |} and have no content restrictions
if (previousChar === "|" && currentChar === "}") {
// Record the marker
const objectMarkerNameText = content.substring(openMarker!.sourcePosition + 2, i - 1).trim();
const marker = recordObjectMarker(fileName, openMarker!, objectMarkerNameText, markerMap, markers);
if (openRanges.length > 0) {
openRanges[openRanges.length - 1].marker = marker;
}
// Set the current start to point to the end of the current marker to ignore its text
lastNormalCharPosition = i + 1;
difference += i + 1 - openMarker!.sourcePosition;
// Reset the state
openMarker = undefined;
state = State.none;
}
break;
case State.inSlashStarMarker:
if (previousChar === "*" && currentChar === "/") {
// Record the marker
// start + 2 to ignore the */, -1 on the end to ignore the * (/ is next)
const markerNameText = content.substring(openMarker!.sourcePosition + 2, i - 1).trim();
const marker = recordMarker(fileName, openMarker!, markerNameText, markerMap, markers);
if (openRanges.length > 0) {
openRanges[openRanges.length - 1].marker = marker;
}
// Set the current start to point to the end of the current marker to ignore its text
flush(openMarker!.sourcePosition);
lastNormalCharPosition = i + 1;
difference += i + 1 - openMarker!.sourcePosition;
// Reset the state
openMarker = undefined;
state = State.none;
}
else if (validMarkerChars.indexOf(currentChar) < 0) {
if (currentChar === "*" && i < content.length - 1 && content.charAt(i + 1) === "/") {
// The marker is about to be closed, ignore the 'invalid' char
}
else {
// We've hit a non-valid marker character, so we were actually in a block comment
// Bail out the text we've gathered so far back into the output
flush(i);
lastNormalCharPosition = i;
openMarker = undefined;
state = State.none;
}
}
break;
}
if (currentChar === "\n" && previousChar === "\r") {
// Ignore trailing \n after a \r
continue;
}
else if (currentChar === "\n" || currentChar === "\r") {
line++;
column = 1;
continue;
}
column++;
previousChar = currentChar;
}
}
// Add the remaining text
flush(/*lastSafeCharIndex*/ undefined);
if (openRanges.length > 0) {
const openRange = openRanges[0];
reportError(fileName, openRange.sourceLine, openRange.sourceColumn, "Unterminated range.");
}
if (openMarker) {
reportError(fileName, openMarker.sourceLine, openMarker.sourceColumn, "Unterminated marker.");
}
// put ranges in the correct order
localRanges = localRanges.sort((a, b) => a.pos < b.pos ? -1 : a.pos === b.pos && a.end > b.end ? -1 : 1);
localRanges.forEach((r) => { ranges.push(r); });
return {
content: output,
fileOptions: {},
version: 0,
fileName,
};
}
function chompLeadingSpace(content: string) {
const lines = content.split("\n");
for (const line of lines) {
if ((line.length !== 0) && (line.charAt(0) !== " ")) {
return content;
}
}
return lines.map(s => s.substr(1)).join("\n");
}

126
server/src/tests/harness/fourslash/fourSlashTypes.ts Normal file
View File

@ -0,0 +1,126 @@
/*
* fourSlashTypes.ts
* Copyright (c) Microsoft Corporation.
* Licensed under the MIT license.
*
* Various common types for fourslash test framework
*/
import * as debug from "../../../common/debug";
/** setting file name */
export const pythonSettingFilename = "python.json";
/** well known global option names */
export const enum GlobalMetadataOptionNames {
projectRoot = "projectroot",
ignoreCase = "ignorecase"
}
/** Any option name not belong to this will become global option */
export const enum MetadataOptionNames {
fileName = "filename",
reserved = "reserved"
}
/** List of allowed file metadata names */
export const fileMetadataNames = [MetadataOptionNames.fileName, MetadataOptionNames.reserved];
/** all the necessary information to set the right compiler settings */
export interface CompilerSettings {
[name: string]: string;
}
/** Represents a parsed source file with metadata */
export interface FourSlashFile {
// The contents of the file (with markers, etc stripped out)
content: string;
fileName: string;
version: number;
// File-specific options (name/value pairs)
fileOptions: CompilerSettings;
}
/** Represents a set of parsed source files and options */
export interface FourSlashData {
// Global options (name/value pairs)
globalOptions: CompilerSettings;
files: FourSlashFile[];
// A mapping from marker names to name/position pairs
markerPositions: Map<string, Marker>;
markers: Marker[];
/**
* Inserted in source files by surrounding desired text
* in a range with `[|` and `|]`. For example,
*
* [|text in range|]
*
* is a range with `text in range` "selected".
*/
ranges: Range[];
rangesByText?: MultiMap<Range>;
}
export interface Marker {
fileName: string;
position: number;
data?: {};
}
export interface Range {
fileName: string;
marker?: Marker;
pos: number;
end: number;
}
export interface MultiMap<T> extends Map<string, T[]> {
/**
* Adds the value to an array of values associated with the key, and returns the array.
* Creates the array if it does not already exist.
*/
add(key: string, value: T): T[];
/**
* Removes a value from an array of values associated with the key.
* Does not preserve the order of those values.
* Does nothing if `key` is not in `map`, or `value` is not in `map[key]`.
*/
remove(key: string, value: T): void;
}
/** Review: is this needed? we might just use one from vscode */
export interface HostCancellationToken {
isCancellationRequested(): boolean;
}
export class TestCancellationToken implements HostCancellationToken {
// 0 - cancelled
// >0 - not cancelled
// <0 - not cancelled and value denotes number of isCancellationRequested after which token become cancelled
private static readonly notCanceled = -1;
private numberOfCallsBeforeCancellation = TestCancellationToken.notCanceled;
public isCancellationRequested(): boolean {
if (this.numberOfCallsBeforeCancellation < 0) {
return false;
}
if (this.numberOfCallsBeforeCancellation > 0) {
this.numberOfCallsBeforeCancellation--;
return false;
}
return true;
}
public setCancelled(numberOfCalls = 0): void {
debug.assert(numberOfCalls >= 0);
this.numberOfCallsBeforeCancellation = numberOfCalls;
}
public resetCancelled(): void {
this.numberOfCallsBeforeCancellation = TestCancellationToken.notCanceled;
}
}

59
server/src/tests/harness/fourslash/runner.ts Normal file
View File

@ -0,0 +1,59 @@
/*
* runner.ts
* Copyright (c) Microsoft Corporation.
* Licensed under the MIT license.
*
* Provide APIs to run fourslash tests from provided fourslash markup contents
*/
import * as ts from "typescript";
import { combinePaths } from "../../../common/pathUtils";
import * as host from "../host";
import { parseTestData } from "./fourSlashParser";
import { TestState } from "./testState";
/**
* run given fourslash test file
*
* @param basePath this is used as a base path of the virtual file system the test will run upon
* @param fileName this is the file path where fourslash test file will be read from
*/
export function runFourSlashTest(basePath: string, fileName: string) {
const content = (host.Host.readFile(fileName)!);
runFourSlashTestContent(basePath, fileName, content);
}
/**
* run given fourslash markup content
*
* @param basePath this is used as a base path of the virtual file system the test will run upon
* @param fileName this will be used as a filename of the given `content` in the virtual file system
* if fourslash markup `content` doesn't have explicit `@filename` option
* @param content this is fourslash markup string
*/
export function runFourSlashTestContent(basePath: string, fileName: string, content: string) {
// give file paths an absolute path for the virtual file system
const absoluteBasePath = combinePaths("/", basePath);
const absoluteFileName = combinePaths("/", fileName);
// parse out the files and their metadata
const testData = parseTestData(absoluteBasePath, content, absoluteFileName);
const state = new TestState(absoluteBasePath, testData);
const output = ts.transpileModule(content, { reportDiagnostics: true, compilerOptions: { target: ts.ScriptTarget.ES2015 } });
if (output.diagnostics!.length > 0) {
throw new Error(`Syntax error in ${ absoluteBasePath }: ${ output.diagnostics![0].messageText }`);
}
runCode(output.outputText, state);
}
function runCode(code: string, state: TestState): void {
// Compile and execute the test
const wrappedCode =
`(function(helper) {
${ code }
})`;
const f = eval(wrappedCode);
f(state);
}

715
server/src/tests/harness/fourslash/testState.ts Normal file
View File

@ -0,0 +1,715 @@
/*
* testState.ts
* Copyright (c) Microsoft Corporation.
* Licensed under the MIT license.
*
* TestState wraps currently test states and provides a way to query and manipulate
* the test states.
*/
import * as path from "path";
import * as assert from 'assert'
import Char from "typescript-char";
import { ImportResolver } from "../../../analyzer/importResolver";
import { Program } from "../../../analyzer/program";
import { ConfigOptions } from "../../../common/configOptions";
import { NullConsole } from "../../../common/console";
import { Comparison, isNumber, isString } from "../../../common/core";
import * as debug from "../../../common/debug";
import { DiagnosticCategory } from "../../../common/diagnostic";
import { combinePaths, comparePaths, getBaseFileName, normalizePath, normalizeSlashes } from "../../../common/pathUtils";
import { convertOffsetToPosition, convertPositionToOffset } from "../../../common/positionUtils";
import { getStringComparer } from "../../../common/stringUtils";
import { Position, TextRange } from "../../../common/textRange";
import * as host from "../host";
import { createFromFileSystem } from "../vfs/factory";
import * as vfs from "../vfs/filesystem";
import { CompilerSettings, FourSlashData, FourSlashFile, GlobalMetadataOptionNames, Marker, MultiMap, pythonSettingFilename, Range, TestCancellationToken } from "./fourSlashTypes";
import { stringify } from "../utils"
export interface TextChange {
span: TextRange;
newText: string;
}
export class TestState {
private readonly _cancellationToken: TestCancellationToken;
private readonly _files: string[] = [];
public readonly fs: vfs.FileSystem;
public readonly importResolver: ImportResolver;
public readonly configOptions: ConfigOptions;
public readonly program: Program;
// The current caret position in the active file
public currentCaretPosition = 0;
// The position of the end of the current selection, or -1 if nothing is selected
public selectionEnd = -1;
public lastKnownMarker = "";
// The file that's currently 'opened'
public activeFile!: FourSlashFile;
constructor(private _basePath: string, public testData: FourSlashData) {
const strIgnoreCase = GlobalMetadataOptionNames.ignoreCase;
const ignoreCase = testData.globalOptions[strIgnoreCase]?.toUpperCase() === "TRUE";
this._cancellationToken = new TestCancellationToken();
const configOptions = this._convertGlobalOptionsToConfigOptions(this.testData.globalOptions);
const files: vfs.FileSet = {};
for (const file of testData.files) {
// if one of file is configuration file, set config options from the given json
if (this._isConfig(file, ignoreCase)) {
let configJson: any;
try {
configJson = JSON.parse(file.content);
}
catch (e) {
throw new Error(`Failed to parse test ${ file.fileName }: ${ e.message }`);
}
configOptions.initializeFromJson(configJson, new NullConsole());
}
else {
files[file.fileName] = new vfs.File(file.content, { meta: file.fileOptions, encoding: "utf8" });
}
}
const fs = createFromFileSystem(host.Host, ignoreCase, { cwd: _basePath, files: files, meta: testData.globalOptions });
// this should be change to AnalyzerService rather than Program
const importResolver = new ImportResolver(fs, configOptions);
const program = new Program(importResolver, configOptions);
program.setTrackedFiles(Object.keys(files));
// make sure these states are consistent between these objects.
// later make sure we just hold onto AnalyzerService and get all these
// state from 1 analyzerService so that we always use same consistent states
this.fs = fs;
this.configOptions = configOptions;
this.importResolver = importResolver;
this.program = program;
this._files.push(...Object.keys(files));
if (this._files.length > 0)
{
// Open the first file by default
this.openFile(0);
}
}
// Entry points from fourslash.ts
public goToMarker(nameOrMarker: string | Marker = "") {
const marker = isString(nameOrMarker) ? this.getMarkerByName(nameOrMarker) : nameOrMarker;
if (this.activeFile.fileName !== marker.fileName) {
this.openFile(marker.fileName);
}
const content = this._getFileContent(marker.fileName);
if (marker.position === -1 || marker.position > content.length) {
throw new Error(`Marker "${ nameOrMarker }" has been invalidated by unrecoverable edits to the file.`);
}
const mName = isString(nameOrMarker) ? nameOrMarker : this.getMarkerName(marker);
this.lastKnownMarker = mName;
this.goToPosition(marker.position);
}
public goToEachMarker(markers: readonly Marker[], action: (marker: Marker, index: number) => void) {
debug.assert(markers.length > 0);
for (let i = 0; i < markers.length; i++) {
this.goToMarker(markers[i]);
action(markers[i], i);
}
}
public getMarkerName(m: Marker): string {
let found: string | undefined = undefined;
this.testData.markerPositions.forEach((marker, name) => {
if (marker === m) {
found = name;
}
});
debug.assertDefined(found);
return found!;
}
public getMarkerByName(markerName: string) {
const markerPos = this.testData.markerPositions.get(markerName);
if (markerPos === undefined) {
throw new Error(`Unknown marker "${ markerName }" Available markers: ${ this.getMarkerNames().map(m => "\"" + m + "\"").join(", ") }`);
}
else {
return markerPos;
}
}
public getMarkers(): Marker[] {
// Return a copy of the list
return this.testData.markers.slice(0);
}
public getMarkerNames(): string[] {
return [...this.testData.markerPositions.keys()];
}
public goToPosition(positionOrLineAndColumn: number | Position) {
const pos = isNumber(positionOrLineAndColumn)
? positionOrLineAndColumn
: this._convertPositionToOffset(this.activeFile.fileName, positionOrLineAndColumn);
this.currentCaretPosition = pos;
this.selectionEnd = -1;
}
public select(startMarker: string, endMarker: string) {
const start = this.getMarkerByName(startMarker), end = this.getMarkerByName(endMarker);
debug.assert(start.fileName === end.fileName);
if (this.activeFile.fileName !== start.fileName) {
this.openFile(start.fileName);
}
this.goToPosition(start.position);
this.selectionEnd = end.position;
}
public selectAllInFile(fileName: string) {
this.openFile(fileName);
this.goToPosition(0);
this.selectionEnd = this.activeFile.content.length;
}
public selectRange(range: Range): void {
this.goToRangeStart(range);
this.selectionEnd = range.end;
}
public selectLine(index: number) {
const lineStart = this._convertPositionToOffset(this.activeFile.fileName, { line: index, character: 0 });
const lineEnd = lineStart + this._getLineContent(index).length;
this.selectRange({ fileName: this.activeFile.fileName, pos: lineStart, end: lineEnd });
}
public goToEachRange(action: (range: Range) => void) {
const ranges = this.getRanges();
debug.assert(ranges.length > 0);
for (const range of ranges) {
this.selectRange(range);
action(range);
}
}
public goToRangeStart({ fileName, pos }: Range) {
this.openFile(fileName);
this.goToPosition(pos);
}
public getRanges(): Range[] {
return this.testData.ranges;
}
public getRangesInFile(fileName = this.activeFile.fileName) {
return this.getRanges().filter(r => r.fileName === fileName);
}
public getRangesByText(): Map<string, Range[]> {
if (this.testData.rangesByText) return this.testData.rangesByText;
const result = this._createMultiMap<Range>(this.getRanges(), r => this._rangeText(r));
this.testData.rangesByText = result;
return result;
}
public goToBOF() {
this.goToPosition(0);
}
public goToEOF() {
const len = this._getFileContent(this.activeFile.fileName).length;
this.goToPosition(len);
}
public moveCaretRight(count = 1) {
this.currentCaretPosition += count;
this.currentCaretPosition = Math.min(this.currentCaretPosition, this._getFileContent(this.activeFile.fileName).length);
this.selectionEnd = -1;
}
// Opens a file given its 0-based index or fileName
public openFile(indexOrName: number | string, content?: string): void {
const fileToOpen: FourSlashFile = this._findFile(indexOrName);
fileToOpen.fileName = normalizeSlashes(fileToOpen.fileName);
this.activeFile = fileToOpen;
// Let the host know that this file is now open
// this.languageServiceAdapterHost.openFile(fileToOpen.fileName, content);
}
public printCurrentFileState(showWhitespace: boolean, makeCaretVisible: boolean) {
for (const file of this.testData.files) {
const active = (this.activeFile === file);
host.Host.log(`=== Script (${ file.fileName }) ${ (active ? "(active, cursor at |)" : "") } ===`);
let content = this._getFileContent(file.fileName);
if (active) {
content = content.substr(0, this.currentCaretPosition) + (makeCaretVisible ? "|" : "") + content.substr(this.currentCaretPosition);
}
if (showWhitespace) {
content = this._makeWhitespaceVisible(content);
}
host.Host.log(content);
}
}
public deleteChar(count = 1) {
const offset = this.currentCaretPosition;
const ch = "";
const checkCadence = (count >> 2) + 1;
for (let i = 0; i < count; i++) {
this._editScriptAndUpdateMarkers(this.activeFile.fileName, offset, offset + 1, ch);
if (i % checkCadence === 0) {
this._checkPostEditInvariants();
}
}
this._checkPostEditInvariants();
}
public replace(start: number, length: number, text: string) {
this._editScriptAndUpdateMarkers(this.activeFile.fileName, start, start + length, text);
this._checkPostEditInvariants();
}
public deleteLineRange(startIndex: number, endIndexInclusive: number) {
const startPos = this._convertPositionToOffset(this.activeFile.fileName, { line: startIndex, character: 0 });
const endPos = this._convertPositionToOffset(this.activeFile.fileName, { line: endIndexInclusive + 1, character: 0 });
this.replace(startPos, endPos - startPos, "");
}
public deleteCharBehindMarker(count = 1) {
let offset = this.currentCaretPosition;
const ch = "";
const checkCadence = (count >> 2) + 1;
for (let i = 0; i < count; i++) {
this.currentCaretPosition--;
offset--;
this._editScriptAndUpdateMarkers(this.activeFile.fileName, offset, offset + 1, ch);
if (i % checkCadence === 0) {
this._checkPostEditInvariants();
}
// Don't need to examine formatting because there are no formatting changes on backspace.
}
this._checkPostEditInvariants();
}
// Enters lines of text at the current caret position
public type(text: string) {
let offset = this.currentCaretPosition;
const selection = this._getSelection();
this.replace(selection.start, selection.length, "");
for (let i = 0; i < text.length; i++) {
const ch = text.charAt(i);
this._editScriptAndUpdateMarkers(this.activeFile.fileName, offset, offset, ch);
this.currentCaretPosition++;
offset++;
}
this._checkPostEditInvariants();
}
// Enters text as if the user had pasted it
public paste(text: string) {
this._editScriptAndUpdateMarkers(this.activeFile.fileName, this.currentCaretPosition, this.currentCaretPosition, text);
this._checkPostEditInvariants();
}
public verifyDiagnostics(map?: { [marker: string]: { category: string; message: string } }): void {
while (this.program.analyze()) {
// Continue to call analyze until it completes. Since we're not
// specifying a timeout, it should complete the first time.
}
const sourceFiles = this._files.map(f => this.program.getSourceFile(f));
const results = sourceFiles.map((sourceFile, index) => {
if (sourceFile) {
const diagnostics = sourceFile.getDiagnostics(this.configOptions) || [];
const filePath = sourceFile.getFilePath();
const value = {
filePath: filePath,
parseResults: sourceFile.getParseResults(),
errors: diagnostics.filter(diag => diag.category === DiagnosticCategory.Error),
warnings: diagnostics.filter(diag => diag.category === DiagnosticCategory.Warning)
};
return [filePath, value] as [string, typeof value];
} else {
this._raiseError(`Source file not found for ${ this._files[index] }`);
}
});
// organize things per file
const resultPerFile = new Map<string, typeof results[0][1]>(results);
const rangePerFile = this._createMultiMap<Range>(this.getRanges(), r => r.fileName);
// expected number of files
if (resultPerFile.size != rangePerFile.size) {
this._raiseError(`actual and expected doesn't match - expected: ${ stringify(rangePerFile) }, actual: ${ stringify(rangePerFile) }`);
}
for (const [file, ranges] of rangePerFile.entries()) {
const rangesPerCategory = this._createMultiMap<Range>(ranges, r => {
if (map) {
const name = this.getMarkerName(r.marker!);
return map[name].category;
}
return (r.marker!.data! as any).category as string;
});
const result = resultPerFile.get(file)!;
for (const [category, expected] of rangesPerCategory.entries()) {
const lines = result.parseResults!.tokenizerOutput.lines;
const actual = category === "error" ? result.errors : category === "warning" ? result.warnings : this._raiseError(`unexpected category ${ category }`);
if (expected.length !== actual.length) {
this._raiseError(`contains unexpected result - expected: ${ stringify(expected) }, actual: ${ actual }`);
}
for (const range of ranges) {
const rangeSpan = TextRange.fromBounds(range.pos, range.end);
const matches = actual.filter(d => {
const diagnosticSpan = TextRange.fromBounds(convertPositionToOffset(d.range.start, lines)!,
convertPositionToOffset(d.range.end, lines)!);
return this._deepEqual(diagnosticSpan, rangeSpan); });
if (matches.length === 0) {
this._raiseError(`doesn't contain expected range: ${ stringify(range) }`);
}
// if map is provided, check messasge as well
if (map) {
const name = this.getMarkerName(range.marker!);
const message = map[name].message;
if (matches.filter(d => message == d.message).length !== 1) {
this._raiseError(`message doesn't match: ${ message } of ${ name } - ${ stringify(range) }, actual: ${ stringify(matches) }`);
}
}
}
}
}
}
public verifyCaretAtMarker(markerName = "") {
const pos = this.getMarkerByName(markerName);
if (pos.fileName !== this.activeFile.fileName) {
throw new Error(`verifyCaretAtMarker failed - expected to be in file "${ pos.fileName }", but was in file "${ this.activeFile.fileName }"`);
}
if (pos.position !== this.currentCaretPosition) {
throw new Error(`verifyCaretAtMarker failed - expected to be at marker "/*${ markerName }*/, but was at position ${ this.currentCaretPosition }(${ this._getLineColStringAtPosition(this.currentCaretPosition) })`);
}
}
public verifyCurrentLineContent(text: string) {
const actual = this._getCurrentLineContent();
if (actual !== text) {
throw new Error("verifyCurrentLineContent\n" + this._displayExpectedAndActualString(text, actual, /* quoted */ true));
}
}
public verifyCurrentFileContent(text: string) {
this._verifyFileContent(this.activeFile.fileName, text);
}
public verifyTextAtCaretIs(text: string) {
const actual = this._getFileContent(this.activeFile.fileName).substring(this.currentCaretPosition, this.currentCaretPosition + text.length);
if (actual !== text) {
throw new Error("verifyTextAtCaretIs\n" + this._displayExpectedAndActualString(text, actual, /* quoted */ true));
}
}
public verifyRangeIs(expectedText: string, includeWhiteSpace?: boolean) {
this._verifyTextMatches(this._rangeText(this._getOnlyRange()), !!includeWhiteSpace, expectedText);
}
public setCancelled(numberOfCalls: number): void {
this._cancellationToken.setCancelled(numberOfCalls);
}
public resetCancelled(): void {
this._cancellationToken.resetCancelled();
}
private _isConfig(file: FourSlashFile, ignoreCase: boolean): boolean {
const comparer = getStringComparer(ignoreCase);
return comparer(getBaseFileName(file.fileName), pythonSettingFilename) == Comparison.EqualTo;
}
private _convertGlobalOptionsToConfigOptions(globalOptions: CompilerSettings): ConfigOptions {
const srtRoot: string = GlobalMetadataOptionNames.projectRoot;
const projectRoot = normalizeSlashes(globalOptions[srtRoot] ?? ".");
const configOptions = new ConfigOptions(projectRoot);
// add more global options as we need them
// Always enable "test mode".
configOptions.internalTestMode = true;
return configOptions;
}
private _getFileContent(fileName: string): string {
const files = this.testData.files.filter(f => comparePaths(f.fileName, fileName, this.fs.ignoreCase) === Comparison.EqualTo);
return files[0].content;
}
private _convertPositionToOffset(fileName: string, position: Position): number {
const result = this._getParseResult(fileName);
return convertPositionToOffset(position, result.tokenizerOutput.lines)!;
}
private _convertOffsetToPosition(fileName: string, offset: number): Position {
const result = this._getParseResult(fileName);
return convertOffsetToPosition(offset, result.tokenizerOutput.lines);
}
private _getParseResult(fileName: string) {
const file = this.program.getSourceFile(fileName)!;
file.parse(this.configOptions, this.importResolver);
return file.getParseResults()!;
}
private _raiseError(message: string): never {
throw new Error(this._messageAtLastKnownMarker(message));
}
private _messageAtLastKnownMarker(message: string) {
const locationDescription = this.lastKnownMarker ? this.lastKnownMarker : this._getLineColStringAtPosition(this.currentCaretPosition);
return `At ${ locationDescription }: ${ message }`;
}
private _checkPostEditInvariants() {
// blank for now
}
private _editScriptAndUpdateMarkers(fileName: string, editStart: number, editEnd: number, newText: string) {
// this.languageServiceAdapterHost.editScript(fileName, editStart, editEnd, newText);
for (const marker of this.testData.markers) {
if (marker.fileName === fileName) {
marker.position = this._updatePosition(marker.position, editStart, editEnd, newText);
}
}
for (const range of this.testData.ranges) {
if (range.fileName === fileName) {
range.pos = this._updatePosition(range.pos, editStart, editEnd, newText);
range.end = this._updatePosition(range.end, editStart, editEnd, newText);
}
}
this.testData.rangesByText = undefined;
}
private _removeWhitespace(text: string): string {
return text.replace(/\s/g, "");
}
private _createMultiMap<T>(values?: T[], getKey?: (t: T) => string): MultiMap<T> {
const map = new Map<string, T[]>() as MultiMap<T>;
map.add = multiMapAdd;
map.remove = multiMapRemove;
if (values && getKey) {
for (const value of values) {
map.add(getKey(value), value);
}
}
return map;
function multiMapAdd<T>(this: MultiMap<T>, key: string, value: T) {
let values = this.get(key);
if (values) {
values.push(value);
}
else {
this.set(key, values = [value]);
}
return values;
}
function multiMapRemove<T>(this: MultiMap<T>, key: string, value: T) {
const values = this.get(key);
if (values) {
values.forEach((v, i, arr) => { if (v === value) arr.splice(i, 1) });
if (!values.length) {
this.delete(key);
}
}
}
}
private _rangeText({ fileName, pos, end }: Range): string {
return this._getFileContent(fileName).slice(pos, end);
}
private _getOnlyRange() {
const ranges = this.getRanges();
if (ranges.length !== 1) {
this._raiseError("Exactly one range should be specified in the testfile.");
}
return ranges[0];
}
private _verifyFileContent(fileName: string, text: string) {
const actual = this._getFileContent(fileName);
if (actual !== text) {
throw new Error(`verifyFileContent failed:\n${ this._showTextDiff(text, actual) }`);
}
}
private _verifyTextMatches(actualText: string, includeWhitespace: boolean, expectedText: string) {
const removeWhitespace = (s: string): string => includeWhitespace ? s : this._removeWhitespace(s);
if (removeWhitespace(actualText) !== removeWhitespace(expectedText)) {
this._raiseError(`Actual range text doesn't match expected text.\n${ this._showTextDiff(expectedText, actualText) }`);
}
}
private _getSelection(): TextRange {
return TextRange.fromBounds(this.currentCaretPosition, this.selectionEnd === -1 ? this.currentCaretPosition : this.selectionEnd);
}
private _getLineContent(index: number) {
const text = this._getFileContent(this.activeFile.fileName);
const pos = this._convertPositionToOffset(this.activeFile.fileName, { line: index, character: 0 });
let startPos = pos, endPos = pos;
while (startPos > 0) {
const ch = text.charCodeAt(startPos - 1);
if (ch === Char.CarriageReturn || ch === Char.LineFeed) {
break;
}
startPos--;
}
while (endPos < text.length) {
const ch = text.charCodeAt(endPos);
if (ch === Char.CarriageReturn || ch === Char.LineFeed) {
break;
}
endPos++;
}
return text.substring(startPos, endPos);
}
// Get the text of the entire line the caret is currently at
private _getCurrentLineContent() {
return this._getLineContent(this._convertOffsetToPosition(
this.activeFile.fileName,
this.currentCaretPosition,
).line);
}
private _findFile(indexOrName: string | number): FourSlashFile {
if (typeof indexOrName === "number") {
const index = indexOrName;
if (index >= this.testData.files.length) {
throw new Error(`File index (${ index }) in openFile was out of range. There are only ${ this.testData.files.length } files in this test.`);
}
else {
return this.testData.files[index];
}
}
else if (isString(indexOrName)) {
const { file, availableNames } = this._tryFindFileWorker(indexOrName);
if (!file) {
throw new Error(`No test file named "${ indexOrName }" exists. Available file names are: ${ availableNames.join(", ") }`);
}
return file;
}
else {
return debug.assertNever(indexOrName);
}
}
private _tryFindFileWorker(name: string): { readonly file: FourSlashFile | undefined; readonly availableNames: readonly string[] } {
name = normalizePath(name);
// names are stored in the compiler with this relative path, this allows people to use goTo.file on just the fileName
name = name.indexOf(path.sep) === -1 ? combinePaths(this._basePath, name) : name;
let file: FourSlashFile | undefined = undefined;
const availableNames: string[] = [];
this.testData.files.forEach(f => {
const fn = normalizePath(f.fileName);
if (fn) {
if (fn === name) {
file = f;
}
availableNames.push(fn);
}
});
debug.assertDefined(file);
return { file, availableNames };
}
private _getLineColStringAtPosition(position: number, file: FourSlashFile = this.activeFile) {
const pos = this._convertOffsetToPosition(file.fileName, position);
return `line ${ (pos.line + 1) }, col ${ pos.character }`;
}
private _showTextDiff(expected: string, actual: string): string {
// Only show whitespace if the difference is whitespace-only.
if (this._differOnlyByWhitespace(expected, actual)) {
expected = this._makeWhitespaceVisible(expected);
actual = this._makeWhitespaceVisible(actual);
}
return this._displayExpectedAndActualString(expected, actual);
}
private _differOnlyByWhitespace(a: string, b: string) {
return this._removeWhitespace(a) === this._removeWhitespace(b);
}
private _displayExpectedAndActualString(expected: string, actual: string, quoted = false) {
const expectMsg = "\x1b[1mExpected\x1b[0m\x1b[31m";
const actualMsg = "\x1b[1mActual\x1b[0m\x1b[31m";
const expectedString = quoted ? "\"" + expected + "\"" : expected;
const actualString = quoted ? "\"" + actual + "\"" : actual;
return `\n${ expectMsg }:\n${ expectedString }\n\n${ actualMsg }:\n${ actualString }`;
}
private _makeWhitespaceVisible(text: string) {
return text.replace(/ /g, "\u00B7").replace(/\r/g, "\u00B6").replace(/\n/g, "\u2193\n").replace(/\t/g, "\u2192 ");
}
private _updatePosition(position: number, editStart: number, editEnd: number, { length }: string): number {
// If inside the edit, return -1 to mark as invalid
return position <= editStart ? position : position < editEnd ? -1 : position + length - + (editEnd - editStart);
}
private _deepEqual(a: any, e: any) {
try {
// NOTE: find better way.
assert.deepStrictEqual(a, e);
} catch {
return false;
}
return true;
}
}

159
server/src/tests/harness/host.ts Normal file
View File

@ -0,0 +1,159 @@
/*
* io.ts
* Copyright (c) Microsoft Corporation.
* Licensed under the MIT license.
*/
import * as pathModule from "path";
import * as os from "os";
import { compareStringsCaseSensitive, compareStringsCaseInsensitive } from "../../common/stringUtils";
import { directoryExists, FileSystemEntries, combinePaths, fileExists, getFileSize, resolvePaths } from "../../common/pathUtils";
import { createFromRealFileSystem } from '../../common/vfs';
import { NullConsole } from '../../common/console';
export const Host: TestHost = createHost();
// eslint-disable-next-line @typescript-eslint/interface-name-prefix
export interface TestHost {
useCaseSensitiveFileNames(): boolean;
getAccessibleFileSystemEntries(dirname: string): FileSystemEntries;
directoryExists(path: string): boolean;
fileExists(fileName: string): boolean;
getFileSize(path: string): number;
readFile(path: string): string | undefined;
getWorkspaceRoot(): string;
writeFile(path: string, contents: string): void;
listFiles(path: string, filter?: RegExp, options?: {
recursive?: boolean;
}): string[];
log(text: string): void;
}
function createHost(): TestHost {
// NodeJS detects "\uFEFF" at the start of the string and *replaces* it with the actual
// byte order mark from the specified encoding. Using any other byte order mark does
// not actually work.
const byteOrderMarkIndicator = "\uFEFF";
const vfs = createFromRealFileSystem(new NullConsole());
const useCaseSensitiveFileNames = isFileSystemCaseSensitive();
function isFileSystemCaseSensitive(): boolean {
// win32\win64 are case insensitive platforms
const platform = os.platform();
if (platform === "win32") {
return false;
}
// If this file exists under a different case, we must be case-insensitve.
return !vfs.existsSync(swapCase(__filename));
/** Convert all lowercase chars to uppercase, and vice-versa */
function swapCase(s: string): string {
return s.replace(/\w/g, (ch) => {
const up = ch.toUpperCase();
return ch === up ? ch.toLowerCase() : up;
});
}
}
function listFiles(path: string, spec: RegExp, options: { recursive?: boolean } = {}) {
function filesInFolder(folder: string): string[] {
let paths: string[] = [];
for (const file of vfs.readdirSync(folder)) {
const pathToFile = pathModule.join(folder, file);
const stat = vfs.statSync(pathToFile);
if (options.recursive && stat.isDirectory()) {
paths = paths.concat(filesInFolder(pathToFile));
}
else if (stat.isFile() && (!spec || file.match(spec))) {
paths.push(pathToFile);
}
}
return paths;
}
return filesInFolder(path);
}
function getAccessibleFileSystemEntries(dirname: string): FileSystemEntries {
try {
const entries: string[] = vfs.readdirSync(dirname || ".").sort(useCaseSensitiveFileNames ? compareStringsCaseSensitive : compareStringsCaseInsensitive);
const files: string[] = [];
const directories: string[] = [];
for (const entry of entries) {
if (entry === "." || entry === "..") continue;
const name = combinePaths(dirname, entry);
try {
const stat = vfs.statSync(name);
if (!stat) continue;
if (stat.isFile()) {
files.push(entry);
}
else if (stat.isDirectory()) {
directories.push(entry);
}
}
catch { /*ignore*/ }
}
return { files, directories };
}
catch (e) {
return { files: [], directories: [] };
}
}
function readFile(fileName: string, _encoding?: string): string | undefined {
if (!fileExists(vfs, fileName)) {
return undefined;
}
const buffer = vfs.readFileSync(fileName);
let len = buffer.length;
if (len >= 2 && buffer[0] === 0xFE && buffer[1] === 0xFF) {
// Big endian UTF-16 byte order mark detected. Since big endian is not supported by node.js,
// flip all byte pairs and treat as little endian.
len &= ~1; // Round down to a multiple of 2
for (let i = 0; i < len; i += 2) {
const temp = buffer[i];
buffer[i] = buffer[i + 1];
buffer[i + 1] = temp;
}
return buffer.toString("utf16le", 2);
}
if (len >= 2 && buffer[0] === 0xFF && buffer[1] === 0xFE) {
// Little endian UTF-16 byte order mark detected
return buffer.toString("utf16le", 2);
}
if (len >= 3 && buffer[0] === 0xEF && buffer[1] === 0xBB && buffer[2] === 0xBF) {
// UTF-8 byte order mark detected
return buffer.toString("utf8", 3);
}
// Default is UTF-8 with no byte order mark
return buffer.toString("utf8");
}
function writeFile(fileName: string, data: string, writeByteOrderMark?: boolean): void {
// If a BOM is required, emit one
if (writeByteOrderMark) {
data = byteOrderMarkIndicator + data;
}
vfs.writeFileSync(fileName, data, "utf8");
}
return {
useCaseSensitiveFileNames: () => useCaseSensitiveFileNames,
getFileSize: (path: string) => getFileSize(vfs, path),
readFile: path => readFile(path),
writeFile: (path, content) => writeFile(path, content),
fileExists: path => fileExists(vfs, path),
directoryExists: path => directoryExists(vfs, path),
listFiles,
log: s => console.log(s),
getWorkspaceRoot: () => resolvePaths(__dirname, "../../.."),
getAccessibleFileSystemEntries,
};
}

354
server/src/tests/harness/utils.ts Normal file
View File

@ -0,0 +1,354 @@
/*
* utils.ts
* Copyright (c) Microsoft Corporation.
* Licensed under the MIT license.
*/
import { binarySearch, insertAt } from "../../common/collectionUtils";
import { identity } from "../../common/core";
export interface SortOptions<T> {
comparer: (a: T, b: T) => number;
sort: "insertion" | "comparison";
}
export class SortedMap<K, V> {
private _comparer: (a: K, b: K) => number;
private _keys: K[] = [];
private _values: V[] = [];
private _order: number[] | undefined;
private _version = 0;
private _copyOnWrite = false;
constructor(comparer: ((a: K, b: K) => number) | SortOptions<K>, iterable?: Iterable<[K, V]>) {
this._comparer = typeof comparer === "object" ? comparer.comparer : comparer;
this._order = typeof comparer === "object" && comparer.sort === "insertion" ? [] : undefined;
if (iterable) {
const iterator = getIterator(iterable);
try {
for (let i = nextResult(iterator); i; i = nextResult(iterator)) {
const [key, value] = i.value;
this.set(key, value);
}
}
finally {
closeIterator(iterator);
}
}
}
public get size() {
return this._keys.length;
}
public get comparer() {
return this._comparer;
}
public get [Symbol.toStringTag]() {
return "SortedMap";
}
public has(key: K) {
return binarySearch(this._keys, key, identity, this._comparer) >= 0;
}
public get(key: K) {
const index = binarySearch(this._keys, key, identity, this._comparer);
return index >= 0 ? this._values[index] : undefined;
}
public set(key: K, value: V) {
const index = binarySearch(this._keys, key, identity, this._comparer);
if (index >= 0) {
this._values[index] = value;
}
else {
this.writePreamble();
insertAt(this._keys, ~index, key);
insertAt(this._values, ~index, value);
if (this._order) insertAt(this._order, ~index, this._version);
this.writePostScript();
}
return this;
}
public delete(key: K) {
const index = binarySearch(this._keys, key, identity, this._comparer);
if (index >= 0) {
this.writePreamble();
this.orderedRemoveItemAt(this._keys, index);
this.orderedRemoveItemAt(this._values, index);
if (this._order) this.orderedRemoveItemAt(this._order, index);
this.writePostScript();
return true;
}
return false;
}
public clear() {
if (this.size > 0) {
this.writePreamble();
this._keys.length = 0;
this._values.length = 0;
if (this._order) this._order.length = 0;
this.writePostScript();
}
}
public forEach(callback: (value: V, key: K, collection: this) => void, thisArg?: any) {
const keys = this._keys;
const values = this._values;
const indices = this.getIterationOrder();
const version = this._version;
this._copyOnWrite = true;
try {
if (indices) {
for (const i of indices) {
callback.call(thisArg, values[i], keys[i], this);
}
}
else {
for (let i = 0; i < keys.length; i++) {
callback.call(thisArg, values[i], keys[i], this);
}
}
}
finally {
if (version === this._version) {
this._copyOnWrite = false;
}
}
}
public * keys() {
const keys = this._keys;
const indices = this.getIterationOrder();
const version = this._version;
this._copyOnWrite = true;
try {
if (indices) {
for (const i of indices) {
yield keys[i];
}
}
else {
yield* keys;
}
}
finally {
if (version === this._version) {
this._copyOnWrite = false;
}
}
}
public * values() {
const values = this._values;
const indices = this.getIterationOrder();
const version = this._version;
this._copyOnWrite = true;
try {
if (indices) {
for (const i of indices) {
yield values[i];
}
}
else {
yield* values;
}
}
finally {
if (version === this._version) {
this._copyOnWrite = false;
}
}
}
public * entries() {
const keys = this._keys;
const values = this._values;
const indices = this.getIterationOrder();
const version = this._version;
this._copyOnWrite = true;
try {
if (indices) {
for (const i of indices) {
yield [keys[i], values[i]] as [K, V];
}
}
else {
for (let i = 0; i < keys.length; i++) {
yield [keys[i], values[i]] as [K, V];
}
}
}
finally {
if (version === this._version) {
this._copyOnWrite = false;
}
}
}
public [Symbol.iterator]() {
return this.entries();
}
private writePreamble() {
if (this._copyOnWrite) {
this._keys = this._keys.slice();
this._values = this._values.slice();
if (this._order) this._order = this._order.slice();
this._copyOnWrite = false;
}
}
private writePostScript() {
this._version++;
}
private getIterationOrder() {
if (this._order) {
const order = this._order;
return this._order
.map((_, i) => i)
.sort((x, y) => order[x] - order[y]);
}
return undefined;
}
/** Remove an item by index from an array, moving everything to its right one space left. */
private orderedRemoveItemAt<T>(array: T[], index: number): void {
// This seems to be faster than either `array.splice(i, 1)` or `array.copyWithin(i, i+ 1)`.
for (let i = index; i < array.length - 1; i++) {
array[i] = array[i + 1];
}
array.pop();
}
}
export function getIterator<T>(iterable: Iterable<T>): Iterator<T> {
return iterable[Symbol.iterator]();
}
export function nextResult<T>(iterator: Iterator<T>): IteratorResult<T> | undefined {
const result = iterator.next();
return result.done ? undefined : result;
}
export function closeIterator<T>(iterator: Iterator<T>) {
const fn = iterator.return;
if (typeof fn === "function") fn.call(iterator);
}
/**
* A collection of metadata that supports inheritance.
*/
export class Metadata {
private static readonly _undefinedValue = {};
private _parent: Metadata | undefined;
private _map: { [key: string]: any };
private _version = 0;
private _size = -1;
private _parentVersion: number | undefined;
constructor(parent?: Metadata) {
this._parent = parent;
this._map = Object.create(parent ? parent._map : null);
}
public get size(): number {
if (this._size === -1 || (this._parent && this._parent._version !== this._parentVersion)) {
let size = 0;
for (const _ in this._map) size++;
this._size = size;
if (this._parent) {
this._parentVersion = this._parent._version;
}
}
return this._size;
}
public get parent() {
return this._parent;
}
public has(key: string): boolean {
return this._map[Metadata._escapeKey(key)] !== undefined;
}
public get(key: string): any {
const value = this._map[Metadata._escapeKey(key)];
return value === Metadata._undefinedValue ? undefined : value;
}
public set(key: string, value: any): this {
this._map[Metadata._escapeKey(key)] = value === undefined ? Metadata._undefinedValue : value;
this._size = -1;
this._version++;
return this;
}
public delete(key: string): boolean {
const escapedKey = Metadata._escapeKey(key);
if (this._map[escapedKey] !== undefined) {
delete this._map[escapedKey];
this._size = -1;
this._version++;
return true;
}
return false;
}
public clear(): void {
this._map = Object.create(this._parent ? this._parent._map : null);
this._size = -1;
this._version++;
}
public forEach(callback: (value: any, key: string, map: this) => void) {
for (const key in this._map) {
callback(this._map[key], Metadata._unescapeKey(key), this);
}
}
private static _escapeKey(text: string) {
return (text.length >= 2 && text.charAt(0) === "_" && text.charAt(1) === "_" ? "_" + text : text);
}
private static _unescapeKey(text: string) {
return (text.length >= 3 && text.charAt(0) === "_" && text.charAt(1) === "_" && text.charAt(2) === "_" ? text.slice(1) : text);
}
}
export function bufferFrom(input: string, encoding?: BufferEncoding): Buffer {
// See https://github.com/Microsoft/TypeScript/issues/25652
return Buffer.from && (Buffer.from as Function) !== Int8Array.from
? Buffer.from(input, encoding) : new Buffer(input, encoding);
}
export const IOErrorMessages = Object.freeze({
EACCES: "access denied",
EIO: "an I/O error occurred",
ENOENT: "no such file or directory",
EEXIST: "file already exists",
ELOOP: "too many symbolic links encountered",
ENOTDIR: "no such directory",
EISDIR: "path is a directory",
EBADF: "invalid file descriptor",
EINVAL: "invalid value",
ENOTEMPTY: "directory not empty",
EPERM: "operation not permitted",
EROFS: "file system is read-only"
});
export function createIOError(code: keyof typeof IOErrorMessages, details = "") {
const err: NodeJS.ErrnoException = new Error(`${ code }: ${ IOErrorMessages[code] } ${ details }`);
err.code = code;
if (Error.captureStackTrace) Error.captureStackTrace(err, createIOError);
return err;
}
export function stringify(data: any, replacer?: (key: string, value: any) => any): string {
return JSON.stringify(data, replacer, 2);
}

140
server/src/tests/harness/vfs/factory.ts Normal file
View File

@ -0,0 +1,140 @@
/*
* factory.ts
* Copyright (c) Microsoft Corporation.
* Licensed under the MIT license.
*
* Provides a factory to create virtual file system backed by a real file system with some path remapped
*/
import { combinePaths, getDirectoryPath, normalizeSlashes, resolvePaths } from "../../../common/pathUtils";
import { TestHost } from "../host";
import { bufferFrom } from "../utils";
import { FileSystem, FileSystemOptions, FileSystemResolver, ModulePath, Mount, S_IFDIR, S_IFREG } from "./filesystem";
export class TextDocument {
public readonly meta: Map<string, string>;
public readonly file: string;
public readonly text: string;
constructor(file: string, text: string, meta?: Map<string, string>) {
this.file = file;
this.text = text;
this.meta = meta || new Map<string, string>();
}
}
export interface FileSystemCreateOptions extends FileSystemOptions {
// Sets the documents to add to the file system.
documents?: readonly TextDocument[];
}
export const typeshedFolder = combinePaths(ModulePath, normalizeSlashes("typeshed-fallback"));
export const srcFolder = normalizeSlashes("/.src");
/**
* Create a virtual file system from a physical file system using the following path mappings:
*
* - `/typeshed-fallback` is a directory mapped to `${workspaceRoot}/../dist/typeshed-fallback`
* - `/.src` is a virtual directory to be used for tests.
*
* @param host it provides an access to host (real) file system
* @param ignoreCase indicates whether we should ignore casing on this file system or not
* @param documents initial documents to create in this virtual file system
* @param files initial files to create in this virtual file system
* @param cwd initial current working directory in this virtual file system
* @param time initial time in this virtual file system
* @param meta initial metadata in this virtual file system
*
* all `FileSystemCreateOptions` are optional
*/
export function createFromFileSystem(host: TestHost, ignoreCase: boolean, { documents, files, cwd, time, meta }: FileSystemCreateOptions = {}) {
const fs = getBuiltLocal(host, meta ? meta[typeshedFolder] : undefined, ignoreCase).shadow();
if (meta) {
for (const key of Object.keys(meta)) {
fs.meta.set(key, meta[key]);
}
}
if (time) {
fs.time(time);
}
if (cwd) {
fs.mkdirpSync(cwd);
fs.chdir(cwd);
}
if (documents) {
for (const document of documents) {
fs.mkdirpSync(getDirectoryPath(document.file));
fs.writeFileSync(document.file, document.text, "utf8");
fs.filemeta(document.file).set("document", document);
// Add symlinks
const symlink = document.meta.get("symlink");
if (symlink) {
for (const link of symlink.split(",").map(link => link.trim())) {
fs.mkdirpSync(getDirectoryPath(link));
fs.symlinkSync(resolvePaths(fs.cwd(), document.file), link);
}
}
}
}
if (files) {
fs.apply(files);
}
return fs;
}
let cacheKey: { host: TestHost; typeshedFolderPath: string | undefined } | undefined;
let localCIFSCache: FileSystem | undefined;
let localCSFSCache: FileSystem | undefined;
function getBuiltLocal(host: TestHost, typeshedFolderPath: string | undefined, ignoreCase: boolean): FileSystem {
if (cacheKey?.host !== host || cacheKey.typeshedFolderPath != typeshedFolderPath) {
localCIFSCache = undefined;
localCSFSCache = undefined;
cacheKey = { host, typeshedFolderPath };
}
if (!localCIFSCache) {
const resolver = createResolver(host);
typeshedFolderPath = typeshedFolderPath ?? resolvePaths(host.getWorkspaceRoot(), "../client/typeshed-fallback");
localCIFSCache = new FileSystem(/*ignoreCase*/ true, {
files: {
[typeshedFolder]: new Mount(typeshedFolderPath, resolver),
[srcFolder]: {}
},
cwd: srcFolder,
meta: {}
});
localCIFSCache.makeReadonly();
}
if (ignoreCase) return localCIFSCache;
if (!localCSFSCache) {
localCSFSCache = localCIFSCache.shadow(/*ignoreCase*/ false);
localCSFSCache.makeReadonly();
}
return localCSFSCache;
}
function createResolver(host: TestHost): FileSystemResolver {
return {
readdirSync(path: string): string[] {
const { files, directories } = host.getAccessibleFileSystemEntries(path);
return directories.concat(files);
},
statSync(path: string): { mode: number; size: number } {
if (host.directoryExists(path)) {
return { mode: S_IFDIR | 0o777, size: 0 };
}
else if (host.fileExists(path)) {
return { mode: S_IFREG | 0o666, size: host.getFileSize(path) };
}
else {
throw new Error("ENOENT: path does not exist");
}
},
readFileSync(path: string): Buffer {
return bufferFrom!(host.readFile(path)!, "utf8") as Buffer;
}
};
}

1458
server/src/tests/harness/vfs/filesystem.ts Normal file
View File

File diff suppressed because it is too large Load Diff

96
server/src/tests/harness/vfs/pathValidation.ts Normal file
View File

@ -0,0 +1,96 @@
/*
* pathUtils.ts
* Copyright (c) Microsoft Corporation.
* Licensed under the MIT license.
*/
import { sep } from "path"
import * as pu from "../../../common/pathUtils"
import { createIOError } from "../utils";
const invalidRootComponentRegExp = getInvalidRootComponentRegExp();
const invalidNavigableComponentRegExp = /[:*?"<>|]/;
const invalidNavigableComponentWithWildcardsRegExp = /[:"<>|]/;
const invalidNonNavigableComponentRegExp = /^\.{1,2}$|[:*?"<>|]/;
const invalidNonNavigableComponentWithWildcardsRegExp = /^\.{1,2}$|[:"<>|]/;
const extRegExp = /\.\w+$/;
export const enum ValidationFlags {
None = 0,
RequireRoot = 1 << 0,
RequireDirname = 1 << 1,
RequireBasename = 1 << 2,
RequireExtname = 1 << 3,
RequireTrailingSeparator = 1 << 4,
AllowRoot = 1 << 5,
AllowDirname = 1 << 6,
AllowBasename = 1 << 7,
AllowExtname = 1 << 8,
AllowTrailingSeparator = 1 << 9,
AllowNavigation = 1 << 10,
AllowWildcard = 1 << 11,
/** Path must be a valid directory root */
Root = RequireRoot | AllowRoot | AllowTrailingSeparator,
/** Path must be a absolute */
Absolute = RequireRoot | AllowRoot | AllowDirname | AllowBasename | AllowExtname | AllowTrailingSeparator | AllowNavigation,
/** Path may be relative or absolute */
RelativeOrAbsolute = AllowRoot | AllowDirname | AllowBasename | AllowExtname | AllowTrailingSeparator | AllowNavigation,
/** Path may only be a filename */
Basename = RequireBasename | AllowExtname,
}
function validateComponents(components: string[], flags: ValidationFlags, hasTrailingSeparator: boolean) {
const hasRoot = !!components[0];
const hasDirname = components.length > 2;
const hasBasename = components.length > 1;
const hasExtname = hasBasename && extRegExp.test(components[components.length - 1]);
const invalidComponentRegExp = flags & ValidationFlags.AllowNavigation
? flags & ValidationFlags.AllowWildcard ? invalidNavigableComponentWithWildcardsRegExp : invalidNavigableComponentRegExp
: flags & ValidationFlags.AllowWildcard ? invalidNonNavigableComponentWithWildcardsRegExp : invalidNonNavigableComponentRegExp;
// Validate required components
if (flags & ValidationFlags.RequireRoot && !hasRoot) return false;
if (flags & ValidationFlags.RequireDirname && !hasDirname) return false;
if (flags & ValidationFlags.RequireBasename && !hasBasename) return false;
if (flags & ValidationFlags.RequireExtname && !hasExtname) return false;
if (flags & ValidationFlags.RequireTrailingSeparator && !hasTrailingSeparator) return false;
// Required components indicate allowed components
if (flags & ValidationFlags.RequireRoot) flags |= ValidationFlags.AllowRoot;
if (flags & ValidationFlags.RequireDirname) flags |= ValidationFlags.AllowDirname;
if (flags & ValidationFlags.RequireBasename) flags |= ValidationFlags.AllowBasename;
if (flags & ValidationFlags.RequireExtname) flags |= ValidationFlags.AllowExtname;
if (flags & ValidationFlags.RequireTrailingSeparator) flags |= ValidationFlags.AllowTrailingSeparator;
// Validate disallowed components
if (~flags & ValidationFlags.AllowRoot && hasRoot) return false;
if (~flags & ValidationFlags.AllowDirname && hasDirname) return false;
if (~flags & ValidationFlags.AllowBasename && hasBasename) return false;
if (~flags & ValidationFlags.AllowExtname && hasExtname) return false;
if (~flags & ValidationFlags.AllowTrailingSeparator && hasTrailingSeparator) return false;
// Validate component strings
if (invalidRootComponentRegExp.test(components[0])) return false;
for (let i = 1; i < components.length; i++) {
if (invalidComponentRegExp.test(components[i])) return false;
}
return true;
}
export function validate(path: string, flags: ValidationFlags = ValidationFlags.RelativeOrAbsolute) {
const components = pu.getPathComponents(path);
const trailing = pu.hasTrailingDirectorySeparator(path);
if (!validateComponents(components, flags, trailing)) throw createIOError("ENOENT");
return components.length > 1 && trailing ? pu.combinePathComponents(pu.reducePathComponents(components)) + sep : pu.combinePathComponents(pu.reducePathComponents(components));
}
function getInvalidRootComponentRegExp(): RegExp {
const escapedSeparator = pu.getRegexEscapedSeparator();
return new RegExp(`^(?!(${ escapedSeparator }|${ escapedSeparator }${ escapedSeparator }w+${ escapedSeparator }|[a-zA-Z]:${ escapedSeparator }?|)$)`);
}

16
server/src/tests/parser.test.ts
View File

@ -1,12 +1,12 @@
/*
* parser.test.ts
* Copyright (c) Microsoft Corporation.
* Licensed under the MIT license.
* Author: Eric Traut
*
* Unit tests for Python parser. These are very basic because
* the parser gets lots of exercise in the type checker tests.
*/
* parser.test.ts
* Copyright (c) Microsoft Corporation.
* Licensed under the MIT license.
* Author: Eric Traut
*
* Unit tests for Python parser. These are very basic because
* the parser gets lots of exercise in the type checker tests.
*/
import * as assert from 'assert';

190
server/src/tests/pathUtils.test.ts
View File

@ -1,19 +1,36 @@
/*
* pathUtils.test.ts
* Copyright (c) Microsoft Corporation.
* Licensed under the MIT license.
* Author: Eric Traut
*
* Unit tests for pathUtils module.
*/
* pathUtils.test.ts
* Copyright (c) Microsoft Corporation.
* Licensed under the MIT license.
* Author: Eric Traut
*
* Unit tests for pathUtils module.
*/
import * as assert from 'assert';
import * as path from 'path';
import { combinePaths, ensureTrailingDirectorySeparator, getFileExtension,
import {
combinePaths, ensureTrailingDirectorySeparator, getFileExtension,
getFileName, getPathComponents,
getWildcardRegexPattern, getWildcardRoot, hasTrailingDirectorySeparator, stripFileExtension,
stripTrailingDirectorySeparator } from '../common/pathUtils';
stripTrailingDirectorySeparator,
normalizeSlashes,
getRegexEscapedSeparator,
reducePathComponents,
combinePathComponents,
resolvePaths,
comparePaths,
containsPath,
changeAnyExtension,
getAnyExtensionFromPath,
getBaseFileName,
getRelativePathFromDirectory,
comparePathsCaseSensitive,
comparePathsCaseInsensitive,
isRootedDiskPath
} from '../common/pathUtils';
import { Comparison } from '../common/core';
test('getPathComponents1', () => {
const components = getPathComponents('');
@ -58,13 +75,13 @@ test('getPathComponents5', () => {
test('combinePaths1', () => {
const p = combinePaths('/user', '1', '2', '3');
assert.equal(p, path.join(path.sep, 'user', '1', '2', '3'));
assert.equal(p, normalizeSlashes('/user/1/2/3'));
});
test('ensureTrailingDirectorySeparator1', () => {
const p = ensureTrailingDirectorySeparator('hello');
assert.equal(p, `hello${path.sep}`);
assert.equal(p, normalizeSlashes('hello/'));
});
test('hasTrailingDirectorySeparator1', () => {
@ -99,24 +116,165 @@ test('stripFileExtension', () => {
test('getWildcardRegexPattern1', () => {
const pattern = getWildcardRegexPattern('/users/me', './blah/');
assert.equal(pattern, '/users/me/blah');
const sep = getRegexEscapedSeparator();
assert.equal(pattern, `${sep}users${sep}me${sep}blah`);
});
test('getWildcardRegexPattern2', () => {
const pattern = getWildcardRegexPattern('/users/me', './**/*.py?/');
const sep = getRegexEscapedSeparator();
assert.equal(pattern, '/users/me(/[^/.][^/]*)*?/[^/]*\\.py[^/]');
assert.equal(pattern, `${sep}users${sep}me(${sep}[^${sep}.][^${sep}]*)*?${sep}[^${sep}]*\\.py[^${sep}]`);
});
test('getWildcardRoot1', () => {
const p = getWildcardRoot('/users/me', './blah/');
assert.equal(p, path.join(path.sep, 'users', 'me', 'blah'));
assert.equal(p, normalizeSlashes('/users/me/blah'));
});
test('getWildcardRoot2', () => {
const p = getWildcardRoot('/users/me', './**/*.py?/');
assert.equal(p, path.join(path.sep, 'users', 'me'));
assert.equal(p, normalizeSlashes('/users/me'));
});
test('reducePathComponentsEmpty', () => {
assert.equal(reducePathComponents([]).length, 0);
});
test('reducePathComponents', () => {
assert.deepEqual(reducePathComponents(getPathComponents("/a/b/../c/.")), [path.sep, 'a', 'c']);
});
test('combinePathComponentsEmpty', () => {
assert.equal(combinePathComponents([]), "");
});
test('combinePathComponentsAbsolute', () => {
assert.equal(combinePathComponents(["/", "a", "b"]), normalizeSlashes("/a/b"));
});
test('combinePathComponents', () => {
assert.equal(combinePathComponents(["a", "b"]), normalizeSlashes("a/b"));
});
test('resolvePath1', () => {
assert.equal(resolvePaths("/path", "to", "file.ext"), normalizeSlashes("/path/to/file.ext"));
});
test('resolvePath2', () => {
assert.equal(resolvePaths("/path", "to", "..", "from", "file.ext/"), normalizeSlashes("/path/from/file.ext/"));
});
test('comparePaths1', () => {
assert.equal(comparePaths("/A/B/C", "\\a\\b\\c"), Comparison.LessThan);
});
test('comparePaths2', () => {
assert.equal(comparePaths("/A/B/C", "\\a\\b\\c", true), Comparison.EqualTo);
});
test('comparePaths3', () => {
assert.equal(comparePaths("/A/B/C", "/a/c/../b/./c", true), Comparison.EqualTo);
});
test('comparePaths4', () => {
assert.equal(comparePaths("/a/b/c", "/a/c/../b/./c", "current\\path\\", false), Comparison.EqualTo);
});
test('comparePaths5', () => {
assert.equal(comparePaths("/a/b/c/", "/a/b/c"), Comparison.GreaterThan);
});
test('containsPath1', () => {
assert.equal(containsPath("/a/b/c/", "/a/d/../b/c/./d"), true);
});
test('containsPath2', () => {
assert.equal(containsPath("/", "\\a"), true);
});
test('containsPath3', () => {
assert.equal(containsPath("/a", "/A/B", true), true);
});
test('changeAnyExtension1', () => {
assert.equal(changeAnyExtension("/path/to/file.ext", ".js", [".ext", ".ts"], true), "/path/to/file.js");
});
test('changeAnyExtension2', () => {
assert.equal(changeAnyExtension("/path/to/file.ext", ".js"), "/path/to/file.js");
});
test('changeAnyExtension3', () => {
assert.equal(changeAnyExtension("/path/to/file.ext", ".js", ".ts", false), "/path/to/file.ext");
});
test('changeAnyExtension1', () => {
assert.equal(getAnyExtensionFromPath("/path/to/file.ext"), ".ext");
});
test('changeAnyExtension2', () => {
assert.equal(getAnyExtensionFromPath("/path/to/file.ext", ".ts", true), "");
});
test('changeAnyExtension3', () => {
assert.equal(getAnyExtensionFromPath("/path/to/file.ext", [".ext", ".ts"], true), ".ext");
});
test('getBaseFileName1', () => {
assert.equal(getBaseFileName("/path/to/file.ext"), "file.ext");
});
test('getBaseFileName2', () => {
assert.equal(getBaseFileName("/path/to/"), "to");
});
test('getBaseFileName3', () => {
assert.equal(getBaseFileName("c:/"), "");
});
test('getBaseFileName4', () => {
assert.equal(getBaseFileName("/path/to/file.ext", [".ext"], true), "file");
});
test('getRelativePathFromDirectory1', () => {
assert.equal(getRelativePathFromDirectory("/a", "/a/b/c/d", true), normalizeSlashes("b/c/d"));
});
test('getRelativePathFromDirectory2', () => {
assert.equal(getRelativePathFromDirectory("/a", "/b/c/d", true), normalizeSlashes("../b/c/d"));
});
test('comparePathsCaseSensitive', () => {
assert.equal(comparePathsCaseSensitive("/a/b/C", "/a/b/c"), Comparison.LessThan);
});
test('comparePathsCaseInsensitive', () => {
assert.equal(comparePathsCaseInsensitive("/a/b/C", "/a/b/c"), Comparison.EqualTo);
});
test('isRootedDiskPath1', () => {
assert(isRootedDiskPath(normalizeSlashes("C:/a/b")));
});
test('isRootedDiskPath2', () => {
assert(isRootedDiskPath(normalizeSlashes("/")));
});
test('isRootedDiskPath3', () => {
assert(!isRootedDiskPath(normalizeSlashes("a/b")));
});
test('isDiskPathRoot1', () => {
assert(isRootedDiskPath(normalizeSlashes("/")));
});
test('isDiskPathRoot2', () => {
assert(isRootedDiskPath(normalizeSlashes("c:/")));
});
test('isDiskPathRoot3', () => {
assert(!isRootedDiskPath(normalizeSlashes("c:")));
});

28
server/src/tests/samples/dataclass1.py
View File

@ -1,28 +0,0 @@
# This sample validates the Python 3.7 data class feature.
from typing import NamedTuple, Optional
class Other:
pass
class DataTuple(NamedTuple):
def _m(self):
pass
id: int
aid: Other
valll: str = ''
name: Optional[str] = None
d1 = DataTuple(id=1, aid=Other())
d2 = DataTuple(id=1, aid=Other(), valll='v')
d3 = DataTuple(id=1, aid=Other(), name='hello')
d4 = DataTuple(id=1, aid=Other(), name=None)
id = d1.id
# This should generate an error because the name argument
# is the incorrect type.
d5 = DataTuple(id=1, aid=Other(), name=3)
# This should generate an error because aid is a required
# parameter and is missing an argument here.
d6 = DataTuple(id=1, name=None)

17
server/src/tests/samples/dataclass3.py
View File

@ -1,17 +0,0 @@
# This sample validates the Python 3.7 data class feature, ensuring that
# NamedTuple must be a direct base class.
from typing import NamedTuple
class Parent(NamedTuple):
pass
class DataTuple2(Parent):
id: int
# This should generate an error because DataTuple2 isn't considered
# a data class and won't have the associated __new__ or __init__
# method defined.
data = DataTuple2(id=1)

49
server/src/tests/samples/dataclass4.py
View File

@ -1,49 +0,0 @@
# This sample tests the handling of the @dataclass decorator.
from dataclasses import dataclass, InitVar
@dataclass
class Bar():
bbb: int
ccc: str
aaa = 'string'
bar1 = Bar(bbb=5, ccc='hello')
bar2 = Bar(5, 'hello')
bar3 = Bar(5, 'hello', 'hello2')
print(bar3.bbb)
print(bar3.ccc)
print(bar3.aaa)
# This should generate an error because ddd
# isn't a declared value.
bar = Bar(bbb=5, ddd=5, ccc='hello')
# This should generate an error because the
# parameter types don't match.
bar = Bar('hello', 'goodbye')
# This should generate an error because a parameter
# is missing.
bar = Bar(2)
# This should generate an error because there are
# too many parameters.
bar = Bar(2, 'hello', 'hello', 4)
@dataclass
class Baz1():
bbb: int
aaa = 'string'
# This should generate an error because variables
# with no default cannot come after those with
# defaults.
ccc: str
@dataclass
class Baz2():
aaa: str
ddd: InitVar[int] = 3

47
server/src/tests/samples/dataclass5.py
View File

@ -1,47 +0,0 @@
# This sample tests the handling of the @dataclass decorator
# with a custom __init__.
from dataclasses import dataclass
@dataclass(init=False)
class A:
x: int
x_squared: int
def __init__(self, x: int):
self.x = x
self.x_squared = x ** 2
a = A(3)
@dataclass(init=True)
class B:
x: int
x_squared: int
def __init__(self, x: int):
self.x = x
self.x_squared = x ** 2
b = B(3)
@dataclass()
class C:
x: int
x_squared: int
def __init__(self, x: int):
self.x = x
self.x_squared = x ** 2
c = C(3)
@dataclass(init=False)
class D:
x: int
x_squared: int
# This should generate an error because there is no
# override __init__ method and no synthesized __init__.
d = D(3)

26
server/src/tests/samples/dataclass6.py
View File

@ -1,26 +0,0 @@
# This sample tests the type checker's handling of
# synthesized __init__ and __new__ methods for
# dataclass classes and their subclasses.
from dataclasses import dataclass
@dataclass
class A:
x: int
@dataclass(init)
class B(A):
y: int
def __init__(self, a: A, y: int):
self.__dict__ = a.__dict__
a = A(3)
b = B(a, 5)
# This should generate an error because there is an extra parameter
a = A(3, 4)
# This should generate an error because there is one too few parameters
b = B(a)

51
server/src/tests/samples/dataclass7.py
View File

@ -1,51 +0,0 @@
# This sample tests the analyzer's ability to handline inherited
# data classes.
from dataclasses import dataclass
class C1: ...
class C2: ...
class C3: ...
@dataclass
class DC1:
aa: C1
bb: C2
cc: C3
class NonDC2:
ff: int
@dataclass
class DC2(NonDC2, DC1):
ee: C2
aa: C2
dd: C2
dc2_1 = DC2(C2(), C2(), C3(), C2(), C2())
# This should generate an error because the type
# of parameter aa has been replaced with type C1.
dc2_2 = DC2(C1(), C2(), C3(), C2(), C2())
dc2_3 = DC2(ee=C2(), dd=C2(), aa=C2(), bb=C2(), cc=C3())
@dataclass
class DC3:
aa: C1
bb: C2 = C2()
cc: C3 = C3()
@dataclass
class DC4(DC3):
# This should generate an error because
# previous parameters have default values.
dd: C1
@dataclass
class DC5(DC3):
# This should not generate an error because
# aa replaces aa in DC3, and it's ordered
# before the params with default values.
aa: C2

20
server/src/tests/sourceFile.test.ts
View File

@ -1,22 +1,24 @@
/*
* sourceFile.test.ts
* Copyright (c) Microsoft Corporation.
* Licensed under the MIT license.
* Author: Eric Traut
*
* Unit tests for pyright sourceFile module.
*/
* sourceFile.test.ts
* Copyright (c) Microsoft Corporation.
* Licensed under the MIT license.
* Author: Eric Traut
*
* Unit tests for pyright sourceFile module.
*/
import { ImportResolver } from '../analyzer/importResolver';
import { SourceFile } from '../analyzer/sourceFile';
import { ConfigOptions } from '../common/configOptions';
import { combinePaths } from '../common/pathUtils';
import { createFromRealFileSystem } from '../common/vfs';
test('Empty', () => {
const filePath = combinePaths(process.cwd(), 'tests/samples/test_file1.py');
const sourceFile = new SourceFile(filePath, false, false);
const fs = createFromRealFileSystem();
const sourceFile = new SourceFile(fs, filePath, false, false);
const configOptions = new ConfigOptions(process.cwd());
const importResolver = new ImportResolver(configOptions);
const importResolver = new ImportResolver(fs, configOptions);
sourceFile.parse(configOptions, importResolver);
});

29
server/src/tests/stringUtils.test.ts Normal file
View File

@ -0,0 +1,29 @@
/*
* stringUtils.test.ts
* Copyright (c) Microsoft Corporation.
* Licensed under the MIT license.
*/
import * as assert from 'assert';
import * as utils from '../common/stringUtils';
import * as core from '../common/core';
test('CoreCompareStringsCaseInsensitive1', () => {
assert.equal(utils.compareStringsCaseInsensitive("Hello", "hello"), core.Comparison.EqualTo);
});
test('CoreCompareStringsCaseInsensitive2', () => {
assert.equal(utils.compareStringsCaseInsensitive("Hello", undefined), core.Comparison.GreaterThan);
});
test('CoreCompareStringsCaseInsensitive3', () => {
assert.equal(utils.compareStringsCaseInsensitive(undefined, "hello"), core.Comparison.LessThan);
});
test('CoreCompareStringsCaseInsensitive4', () => {
assert.equal(utils.compareStringsCaseInsensitive(undefined, undefined), core.Comparison.EqualTo);
});
test('CoreCompareStringsCaseSensitive', () => {
assert.equal(utils.compareStringsCaseSensitive("Hello", "hello"), core.Comparison.LessThan);
});

536
server/src/tests/testState.test.ts Normal file
View File

@ -0,0 +1,536 @@
/*
* testState.test.ts
* Copyright (c) Microsoft Corporation.
* Licensed under the MIT license.
*
* Tests and show how to use TestState in unit test
*/
import * as assert from 'assert';
import * as factory from "./harness/vfs/factory"
import { normalizeSlashes, combinePaths, comparePathsCaseSensitive } from '../common/pathUtils';
import { parseTestData } from './harness/fourslash/fourSlashParser';
import { TestState } from './harness/fourslash/testState';
import { compareStringsCaseSensitive } from '../common/stringUtils';
import { Range } from './harness/fourslash/fourSlashTypes';
import { runFourSlashTestContent } from './harness/fourslash/runner';
test('Create', () => {
const code = `
// @filename: file1.py
////class A:
//// pass
`
const { data, state } = parseAndGetTestState(code);
assert(state.activeFile === data.files[0]);
});
test('Multiple files', () => {
const code = `
// @filename: file1.py
////class A:
//// pass
// @filename: file2.py
////class B:
//// pass
// @filename: file3.py
////class C:
//// pass
`
const state = parseAndGetTestState(code).state;
assert.equal(state.fs.cwd(), normalizeSlashes("/"));
assert(state.fs.existsSync(normalizeSlashes(combinePaths(factory.srcFolder, "file1.py"))));
});
test('Configuration', () => {
const code = `
// @filename: python.json
//// {
//// "include": [
//// "src"
//// ],
////
//// "exclude": [
//// "**/node_modules",
//// "**/__pycache__",
//// "src/experimental",
//// "src/web/node_modules",
//// "src/typestubs"
//// ],
////
//// "ignore": [
//// "src/oldstuff"
//// ],
////
//// "typingsPath": "src/typestubs",
//// "venvPath": "/home/foo/.venvs",
////
//// "reportTypeshedErrors": false,
//// "reportMissingImports": true,
//// "reportMissingTypeStubs": false,
////
//// "pythonVersion": "3.6",
//// "pythonPlatform": "Linux",
////
//// "executionEnvironments": [
//// {
//// "root": "src/web",
//// "pythonVersion": "3.5",
//// "pythonPlatform": "Windows",
//// "extraPaths": [
//// "src/service_libs"
//// ]
//// },
//// {
//// "root": "src/sdk",
//// "pythonVersion": "3.0",
//// "extraPaths": [
//// "src/backend"
//// ],
//// "venv": "venv_bar"
//// },
//// {
//// "root": "src/tests",
//// "extraPaths": [
//// "src/tests/e2e",
//// "src/sdk"
//// ]
//// },
//// {
//// "root": "src"
//// }
//// ]
//// }
// @filename: file1.py
////class A:
//// pass
`
const state = parseAndGetTestState(code).state;
assert.equal(state.fs.cwd(), normalizeSlashes("/"));
assert(state.fs.existsSync(normalizeSlashes(combinePaths(factory.srcFolder, "file1.py"))));
assert.equal(state.configOptions.diagnosticSettings.reportMissingImports, "error");
assert.equal(state.configOptions.typingsPath, normalizeSlashes("src/typestubs"));
});
test('ProjectRoot', () => {
const code = `
// global options
// @projectRoot: /root
// @filename: /root/file1.py
////class A:
//// pass
`
const state = parseAndGetTestState(code).state;
assert.equal(state.fs.cwd(), normalizeSlashes("/"));
assert(state.fs.existsSync(normalizeSlashes("/root/file1.py")));
assert.equal(state.configOptions.projectRoot, normalizeSlashes("/root"));
});
test('IgnoreCase', () => {
const code = `
// global options
// @ignoreCase: true
// @filename: file1.py
////class A:
//// pass
`
const state = parseAndGetTestState(code).state;
assert(state.fs.existsSync(normalizeSlashes(combinePaths(factory.srcFolder, "FILE1.py"))));
});
test('GoToMarker', () => {
const code = `
////class A:
//// /*marker1*/pass
`
const { data, state } = parseAndGetTestState(code);
const marker = data.markerPositions.get("marker1");
state.goToMarker("marker1");
assert.equal(state.lastKnownMarker, "marker1");
assert.equal(state.currentCaretPosition, marker!.position);
state.goToMarker(marker);
assert.equal(state.lastKnownMarker, "marker1");
assert.equal(state.currentCaretPosition, marker!.position);
assert.equal(state.selectionEnd, -1);
});
test('GoToEachMarker', () => {
const code = `
// @filename: file1.py
////class A:
//// /*marker1*/pass
// @filename: file2.py
////class B:
//// /*marker2*/pass
`
const { data, state } = parseAndGetTestState(code);
const marker1 = data.markerPositions.get("marker1");
const marker2 = data.markerPositions.get("marker2");
const results: number[] = [];
state.goToEachMarker([marker1!, marker2!], m => {
results.push(m.position);
})
assert.deepEqual(results, [marker1!.position, marker2!.position]);
assert.equal(state.activeFile.fileName, marker2!.fileName);
assert.equal(state.currentCaretPosition, marker2!.position);
assert.equal(state.selectionEnd, -1);
});
test('Markers', () => {
const code = `
// @filename: file1.py
////class A:
//// /*marker1*/pass
// @filename: file2.py
////class B:
//// /*marker2*/pass
`
const { data, state } = parseAndGetTestState(code);
const marker1 = data.markerPositions.get("marker1");
assert.deepEqual(state.getMarkerName(marker1!), "marker1");
assert.deepEqual(state.getMarkers().map(m => state.getMarkerName(m)).sort(compareStringsCaseSensitive), state.getMarkerNames().sort(comparePathsCaseSensitive));
});
test('GoToPosition', () => {
const code = `
// @filename: file1.py
////class A:
//// /*marker1*/pass
`
const { data, state } = parseAndGetTestState(code);
const marker1 = data.markerPositions.get("marker1");
state.goToPosition(marker1!.position);
assert.equal(state.currentCaretPosition, marker1!.position);
assert.equal(state.selectionEnd, -1);
});
test('select', () => {
const code = `
// @filename: file1.py
/////*start*/class A:
//// class B:
//// def Test(self):
//// pass
////
//// def Test2(self):
//// pass/*end*/
`
const { data, state } = parseAndGetTestState(code);
state.select("start", "end");
assert.equal(state.currentCaretPosition, data.markerPositions.get("start")!.position);
assert.equal(state.selectionEnd, data.markerPositions.get("end")!.position);
});
test('selectAllInFile', () => {
const code = `
// @filename: file1.py
/////*start*/class A:
//// class B:
//// def Test(self):
//// pass
////
//// def Test2(self):
//// pass/*end*/
`
const { data, state } = parseAndGetTestState(code);
state.selectAllInFile(data.files[0].fileName);
assert.equal(state.currentCaretPosition, data.markerPositions.get("start")!.position);
assert.equal(state.selectionEnd, data.markerPositions.get("end")!.position);
});
test('selectRange', () => {
const code = `
// @filename: file1.py
/////class A:
//// class B:
//// [|def Test(self):
//// pass|]
////
//// def Test2(self):
//// pass
`
const { data, state } = parseAndGetTestState(code);
const range = data.ranges[0];
state.selectRange(range);
assert.equal(state.activeFile.fileName, range.fileName);
assert.equal(state.currentCaretPosition, range.pos);
assert.equal(state.selectionEnd, range.end);
});
test('selectLine', () => {
const code = `
// @filename: file1.py
/////class A:
//// class B:
////[| def Test(self):|]
//// pass
////
//// def Test2(self):
//// pass
`
const { data, state } = parseAndGetTestState(code);
const range = data.ranges[0];
state.selectLine(2);
assert.equal(state.currentCaretPosition, range.pos);
assert.equal(state.selectionEnd, range.end);
});
test('goToEachRange', () => {
const code = `
// @filename: file1.py
/////class A:
//// class B:
//// [|def Test(self):|]
//// pass
////
//// def Test2(self):
//// [|pass|]
`
const { state } = parseAndGetTestState(code);
const results: Range[] = [];
state.goToEachRange(r => {
assert.equal(state.activeFile.fileName, r.fileName);
results.push(r);
})
assert.deepEqual(results, [state.getRanges()[0], state.getRanges()[1]]);
});
test('getRangesInFile', () => {
const code = `
// @filename: file1.py
/////class A:
//// class B:
//// [|def Test(self):|]
//// pass
// @filename: file2.py
//// def Test2(self):
//// [|pass|]
`
const { data, state } = parseAndGetTestState(code);
assert.deepEqual(state.getRangesInFile(data.files[0].fileName), data.ranges.filter(r => r.fileName === data.files[0].fileName));
});
test('rangesByText', () => {
const code = `
// @filename: file1.py
/////class A:
//// class B:
//// [|def Test(self):|]
//// pass
// @filename: file2.py
//// def Test2(self):
//// [|pass|]
`
const { data, state } = parseAndGetTestState(code);
const map = state.getRangesByText();
assert.deepEqual(map.get("def Test(self):"), [data.ranges[0]]);
assert.deepEqual(map.get("pass"), [data.ranges[1]]);
});
test('moveCaretRight', () => {
const code = `
// @filename: file1.py
/////class A:
//// class B:
//// /*position*/def Test(self):
//// pass
////
//// def Test2(self):
//// pass
`
const { data, state } = parseAndGetTestState(code);
const marker = data.markerPositions.get("position")!;
state.goToBOF();
assert.equal(state.currentCaretPosition, 0);
state.goToEOF();
assert.equal(state.currentCaretPosition, data.files[0].content.length);
state.goToPosition(marker.position);
state.moveCaretRight("def".length);
assert.equal(state.currentCaretPosition, marker.position + "def".length);
assert.equal(state.selectionEnd, -1);
});
test('runFourSlashTestContent', () => {
const code = `
/// <reference path="fourslash.d.ts" />
// @filename: file1.py
//// class A:
//// class B:
//// /*position*/def Test(self):
//// pass
////
//// def Test2(self):
//// pass
helper.getMarkerByName("position");
`
runFourSlashTestContent(normalizeSlashes("/"), "unused.py", code);
});
test('VerifyDiagnosticsTest1', () => {
const code = `
/// <reference path="fourslash.d.ts" />
// @filename: dataclass1.py
//// # This sample validates the Python 3.7 data class feature.
////
//// from typing import NamedTuple, Optional
////
//// class Other:
//// pass
////
//// class DataTuple(NamedTuple):
//// def _m(self):
//// pass
//// id: int
//// aid: Other
//// valll: str = ''
//// name: Optional[str] = None
////
//// d1 = DataTuple(id=1, aid=Other())
//// d2 = DataTuple(id=1, aid=Other(), valll='v')
//// d3 = DataTuple(id=1, aid=Other(), name='hello')
//// d4 = DataTuple(id=1, aid=Other(), name=None)
//// id = d1.id
////
//// # This should generate an error because the name argument
//// # is the incorrect type.
//// d5 = DataTuple(id=1, aid=Other(), name=[|{|"category": "error"|}3|])
////
//// # This should generate an error because aid is a required
//// # parameter and is missing an argument here.
//// d6 = [|{|"category": "error"|}DataTuple(id=1, name=None|])
helper.verifyDiagnostics();
`
runFourSlashTestContent(factory.srcFolder, "unused.py", code);
});
test('VerifyDiagnosticsTest2', () => {
const code = `
/// <reference path="fourslash.ts" />
//// # This sample tests the handling of the @dataclass decorator.
////
//// from dataclasses import dataclass, InitVar
////
//// @dataclass
//// class Bar():
//// bbb: int
//// ccc: str
//// aaa = 'string'
////
//// bar1 = Bar(bbb=5, ccc='hello')
//// bar2 = Bar(5, 'hello')
//// bar3 = Bar(5, 'hello', 'hello2')
//// print(bar3.bbb)
//// print(bar3.ccc)
//// print(bar3.aaa)
////
//// # This should generate an error because ddd
//// # isn't a declared value.
//// bar = Bar(bbb=5, [|/*marker1*/ddd|]=5, ccc='hello')
////
//// # This should generate an error because the
//// # parameter types don't match.
//// bar = Bar([|/*marker2*/'hello'|], 'goodbye')
////
//// # This should generate an error because a parameter
//// # is missing.
//// bar = [|/*marker3*/Bar(2)|]
////
//// # This should generate an error because there are
//// # too many parameters.
//// bar = Bar(2, 'hello', 'hello', [|/*marker4*/4|])
////
////
//// @dataclass
//// class Baz1():
//// bbb: int
//// aaa = 'string'
////
//// # This should generate an error because variables
//// # with no default cannot come after those with
//// # defaults.
//// [|/*marker5*/ccc|]: str
////
//// @dataclass
//// class Baz2():
//// aaa: str
//// ddd: InitVar[int] = 3
helper.verifyDiagnostics({
"marker1": { category: "error", message: "No parameter named 'ddd'" },
"marker2": { category: "error", message: "Argument of type 'Literal['hello']' cannot be assigned to parameter 'bbb' of type 'int'\\n 'str' is incompatible with 'int'" },
"marker3": { category: "error", message: "Argument missing for parameter 'ccc'" },
"marker4": { category: "error", message: "Expected 3 positional arguments" },
"marker5": { category: "error", message: "Data fields without default value cannot appear after data fields with default values" },
});
`
runFourSlashTestContent(factory.srcFolder, "unused.py", code);
});
function parseAndGetTestState(code: string) {
const data = parseTestData(factory.srcFolder, code, "test.py");
const state = new TestState(normalizeSlashes("/"), data);
return { data, state };
}

17
server/src/tests/testUtils.ts
View File

@ -1,11 +1,11 @@
/*
* testUtils.ts
* Copyright (c) Microsoft Corporation.
* Licensed under the MIT license.
* Author: Eric Traut
*
* Utility functions that are common to a bunch of the tests.
*/
* testUtils.ts
* Copyright (c) Microsoft Corporation.
* Licensed under the MIT license.
* Author: Eric Traut
*
* Utility functions that are common to a bunch of the tests.
*/
import * as assert from 'assert';
import * as fs from 'fs';
@ -20,6 +20,7 @@ import { cloneDiagnosticSettings, ConfigOptions, ExecutionEnvironment } from '..
import { Diagnostic, DiagnosticCategory } from '../common/diagnostic';
import { DiagnosticSink, TextRangeDiagnosticSink } from '../common/diagnosticSink';
import { ParseOptions, Parser, ParseResults } from '../parser/parser';
import { createFromRealFileSystem } from '../common/vfs';
// This is a bit gross, but it's necessary to allow the fallback typeshed
// directory to be located when running within the jest environment. This
@ -131,7 +132,7 @@ export function typeAnalyzeSampleFiles(fileNames: string[],
// Always enable "test mode".
configOptions.internalTestMode = true;
const importResolver = new ImportResolver(configOptions);
const importResolver = new ImportResolver(createFromRealFileSystem(), configOptions);
const program = new Program(importResolver, configOptions);
const filePaths = fileNames.map(name => resolveSampleFilePath(name));

20
server/src/tests/tokenizer.test.ts
View File

@ -1,14 +1,14 @@
/*
* tokenizer.test.ts
* Copyright (c) Microsoft Corporation.
* Licensed under the MIT license.
* Author: Eric Traut
*
* Based on code from vscode-python repository:
* https://github.com/Microsoft/vscode-python
*
* Unit tests for Python tokenizer.
*/
* tokenizer.test.ts
* Copyright (c) Microsoft Corporation.
* Licensed under the MIT license.
* Author: Eric Traut
*
* Based on code from vscode-python repository:
* https://github.com/Microsoft/vscode-python
*
* Unit tests for Python tokenizer.
*/
import * as assert from 'assert';

3
server/tsconfig.json
View File

@ -11,8 +11,7 @@
"sourceMap": true,
"lib" : [ "es2016" ],
"outDir": "../client/server",
"composite": true,
"declaration": true
"composite": true
},
"exclude": [
"node_modules"