Please provide a short (approximately 100 word) summary of the following web Content, written in the voice of the original author. If there is anything controversial please highlight the controversy. If there is something surprising, unique, or clever, please highlight that as well. Content:
Title: Lessons from Creating a VSCode Extension with GPT-4
Site: bit.kevinslin.com
Lately, I've been playing around with LLMs to write code. I find that they're great at generating small self-contained snippets. Unfortunately, anything more than that requires a human to evaluate LLM output and come up with suitable follow-up prompts. Most examples of "GPT wrote X" are this - a human serves as a REPL for the LLM, carefully coaxing it to a functional result. This is not to undersell this process - it's remarkable that it works. But can we go further? Can we use an LLM to generate ALL the code for a complex program ALL at once without any human intervention? To test GPT-4's ability to generate a complex program, I prompted it to create a VSCode extension that lets the user adjust the heading level of selected Markdown text. This task requires: - Domain-specific knowledge about how to scaffold and expose a program to VSCode - Mixing multiple languages and platforms: VS Code extensions are written in TypeScript, which requires writing configuration for Typescript, Node.js, and VSCode - Generating multiple files  - Generating scaffolding to debug, build, and run code  For this experiment, I used GPT-4 for all generation purposes. I find it to be the most effective among current-day models. In addition, I make use of the  smol-ai  framework to generate code. smol-ai  description from the README: This is a prototype of a "junior developer" agent (aka smol dev) that scaffolds an entire codebase out for you once you give it a product spec, but does not end the world or overpromise AGI. instead of making and maintaining specific, rigid, one-shot starters, like create-react-app, or create-nextjs-app, this is basically create-anything-app where you develop your scaffolding prompt in a tight loop with your smol dev. I like  smol-ai  because of its simplicity. The entire code generation logic is in a single Python file consisting of three primary functions: generate a list of files that are needed to carry out the prompt (eg. package.json, index.js, ...) generate a list of shared dependencies that are needed to carry out the prompt (eg. axios, react, ...) for each file in the generated file list, generate code that would go into the file, making use of shared dependencies if applicable Note that  smol-ai  also provides other capabilities like the ability to debug errors and generate a prompt based on the current code base. These are not explored in this post. I started with a basic prompt describing my problem statement python main_no_modal.py "a vscode extension that lets the user adjust the heading level of the selected text. it should have three commands: increase heading level, decrease heading level, and set heading level"
 Note that the prompt is short and general. I wanted to see how well GPT could do without feeding in any additional details or domain-specific knowledge beyond what was in the training set. After a few minutes spent generating code,  smol-ai  created the following files: .
├── extension
│   ├── package.json
│   ├── src
│   │   ├── commands
│   │   │   ├── decreaseHeadingLevel.ts
│   │   │   ├── increaseHeadingLevel.ts
│   │   │   └── setHeadingLevel.ts
│   │   └── extension.ts
└── shared_dependencies.md
 Note that  shared_dependencies.md  is created as a best guess of what common dependencies will be. In our case, it created a file with the following contents: the app is: a vscode extension that lets the user adjust the heading level of the selected text. it should have three commands: increase heading level, decrease heading level, and set heading level

the files we have decided to generate are: tsconfig.json, package.json, utils.ts, extension.ts

Shared dependencies:

1. Exported variables:
   - adjustHeadingLevel (from utils.ts)

2. Data schemas:
   - None

3. ID names of DOM elements:
   - None

4. Message names:
   - None

5. Function names:
   - increaseHeadingLevel
   - decreaseHeadingLevel
   - setHeadingLevel

6. Dependencies:
    - None
 extension.ts  is the entry point of a vscode extension. In it, we register the three commands that we specified in the prompt. import * as vscode from 'vscode';
import { increaseHeadingLevel } from './commands/increaseHeadingLevel';
import { decreaseHeadingLevel } from './commands/decreaseHeadingLevel';
import { setHeadingLevel } from './commands/setHeadingLevel';

export function activate(context: vscode.ExtensionContext) {
    context.subscriptions.push(
        vscode.commands.registerCommand('extension.increaseHeadingLevel', increaseHeadingLevel),
        vscode.commands.registerCommand('extension.decreaseHeadingLevel', decreaseHeadingLevel),
        vscode.commands.registerCommand('extension.setHeadingLevel', setHeadingLevel)
    );
}

export function deactivate() {}
 Below is the logic inside of  decreaseHeadingLevel.ts import * as vscode from 'vscode';

export async function decreaseHeadingLevel() {
    const editor = vscode.window.activeTextEditor;
    if (!editor) {
        return;
    }

    const document = editor.document;
    const selection = editor.selection;
    const selectedText = document.getText(selection);

    const headingRegex = /^(#+)\s(.*)$/;
    const match = selectedText.match(headingRegex);

    if (!match) {
        vscode.window.showErrorMessage('Invalid selection. Please select a heading to decrease its level.');
        return;
    }

    const currentLevel = match[1].length;
    if (currentLevel === 1) {
        vscode.window.showErrorMessage('Cannot decrease heading level. Already at the lowest level.');
        return;
    }

    const newLevel = currentLevel - 1;
    const newHeading = '#'.repeat(newLevel) + ' ' + match[2];

    await editor.edit((editBuilder) => {
        editBuilder.replace(selection, newHeading);
    });
}
 The code checks for an active editor and if one exists, gets the selected text and looks for a markdown heading via regex. If a header is found and the current header level isn't already at the lowest level, it decreases the heading level. At first glance, there is nothing amiss with the logic. It executes the command and checks for edge cases. It even provides helpful error messages which already puts it ahead of most human-generated programs... To test this extension, we need to successfully execute the following steps: Install Dependencies Compile Code Run Extension We run into our first issue when trying to install dependencies. $ yarn

Couldn't find any versions for "vscode-test" that matches "^1.6.2"
? Please choose a version of "vscode-test" from this list: (Use arrow keys)
❯ 1.6.1
 An inspection of  package.json  returns the following {
  "name": "adjust-heading-level",
  ...
  "engines": {
    "vscode": "^1.62.0"
  },
  "devDependencies": {
    "@types/node": "^14.17.0",
    "@types/vscode": "^1.62.0",
    "typescript": "^4.4.2",
    "vscode": "^1.1.37",
    "vscode-test": "^1.6.2"
  },
}
 The vscode engine determines the minimum version of vscode. The present-day (as of 2023-05-23) engine version is  1.78 . The  1.62.0  version was  released on October 21st, 2021 This corresponds to GPT4's  knowledge cutoff date : GPT-4 generally lacks knowledge of events that have occurred after the vast majority of its data cut off (September 2021) The  vscode-test  version of  1.6.2  seems suspiciously similar to  1.62  which indicates that GPT likely hallucinated the numbers. In any case, this is easy enough to fix by specifying the correct version number and re-installing -   "vscode-test": "^1.6.2"
+   "vscode-test": "^1.6.1"
 Re-running the install process is successful the second time around. $ yarn

...
[3/5] 🚚  Fetching packages...
[4/5] 🔗  Linking dependencies...
[5/5] 🔨  Building fresh packages...
✨  Done in 4.31s.
 Because typescript is a compiled language, we need to execute a build step to compile the code to javascript. The  package.json  comes with the following scripts: "scripts": {
    "vscode:prepublish": "npm run compile",
    "compile": "tsc -p ./",
    "watch": "tsc -watch -p ./",
    "postinstall": "node ./node_modules/vscode/bin/install",
    "test": "npm run compile && node ./node_modules/vscode/bin/test"
  },
 We can build the code by running the  compile  script. This is where we run into our next issue: $ yarn compile
warning package.json: No license field
warning adjust-heading-level@0.1.0: The engine "vscode" appears to be invalid.
$ tsc -p ./
error TS5057: Cannot find a tsconfig.json file at the specified directory: './'.
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
 Typescript requires a  tsconfig.json  file to compile into javascript. If you remember our initial file layout, there is no  tsconfig.json  file. .
├── extension
│   ├── package.json
│   ├── src
│   │   ├── commands
│   │   │   ├── decreaseHeadingLevel.ts
│   │   │   ├── increaseHeadingLevel.ts
│   │   │   └── setHeadingLevel.ts
│   │   └── extension.ts
└── shared_dependencies.md
 We can remediate this by adding the config and re-building. But now we run into more issues: $ tsc --init
$ yarn compile

src/command<truncated>