Add tokenization and context provider API types

- Implemented window delineation tests for indentation-based tokenization.
- Created tokenizer module with various tokenization strategies including TTokenizer and ApproximateTokenizer.
- Added type definitions for authentication parameters and code citation notifications.
- Introduced context provider API for extensions to supply additional context items to Copilot.
- Defined core types and schemas for position and range.
- Established status types for agent status management in IDEs.

completions-sample-code/lib/src/experiments/telemetryNames.ts

@@ -0,0 +1,10 @@
+/*---------------------------------------------------------------------------------------------
+ *  Copyright (c) Microsoft Corporation. All rights reserved.
+ *  Licensed under the MIT License. See License.txt in the project root for license information.
+ *--------------------------------------------------------------------------------------------*/
+
+export enum ExpServiceTelemetryNames {
+	// these are defined (but not exported) in the code for the tas client, currently here:
+	// https://github.com/microsoft/tas-client/blob/75f8895b15ef5696653cbee134ccae24477b0b94/vscode-tas-client/src/vscode-tas-client/VSCodeTasClient.ts#L67
+	featuresTelemetryPropertyName = 'VSCode.ABExp.Features',
+}