Refactoring a Node.js app with Gemini CLI

A Journey with Gemini AI towards Modularity and Quality

For seasoned software development professionals , the hype surrounding artificial intelligence often overshadows its practical implications. This article charts a meticulous journey towards refactoring a Node.js application, focusing on how iterative prompt engineering and a modular approach transformed a generic tool into a highly refined, maintainable with 100% test coverage. We'll dissect the truth of integrating AI tools, using our own development process as a case study.

TL:DR – Refactoring a Node.js app to ensure code quality and maintainability can take a great deal of time and effort,  but with the help of AI you can achieve impressive results in hours rather than days through an iterative process of prompt engineering. Crucially, this journey transformed a basic app nto a sophisticated one, We'll explore how specific prompt adjustments, HTML sanitization, and a robust testing framework (Jest) were implemented, offering practical insights into building reliable applications with the help of AI.

The Paradox of AI Integration: The Power of Prompt Engineering

As we dive into the depths of integrating AI within application development, it's vital to acknowledge the paradox at play. While AI holds the potential to bolster developer productivity, it is often lauded with an exaggeration that most seasoned professionals in the tech world have learned to approach with a healthy dose of scepticism. Instead of aiming for the next big thing, developers should aim for practical enhancements, weaving AI into the fabric of their development process thoughtfully. Our journey with this application highlighted this: initial attempts yielded generic, often flawed, results. The breakthrough came not from a new AI model, but from meticulous prompt engineering.

Tools and Techniques in Our Refactoring Journey

Our refactoring journey wasn't about adopting every new AI tool, but rather about strategically applying specific techniques and frameworks to solve concrete problems. Key among these were:

• Gemini CLI: Free and open source, Gemini CLI brings Gemini directly into developers’ terminals — with unmatched access for individuals. This is the main AI tool used and you can get it from GitHub.
• Iterative Prompt Engineering: This was perhaps the most impactful 'tool'. We continuously refined the AI's system and user prompts to achieve the desired output. This involved many cycles of trial and error, observing the AI's responses, and adjusting the prompts accordingly.
• Modular Architecture: Breaking down the monolithic parts of our `index.js` into `app.js`, `server.js`, and `index.js` (for CLI) significantly improved maintainability and testability.
• `jsdom` for HTML Sanitization: Recognizing that even well-prompted AI can produce imperfect HTML, we integrated `jsdom` to parse and sanitize any generated content, ensuring valid and clean output.
• `direnv` for Environment Management: For secure and streamlined environment variable handling, we adopted `direnv` with a `.envrc.local` pattern, keeping sensitive API keys out of version control.
• Jest and `hono/testing` for Automated Testing: A robust testing suite was implemented to provide a safety net for all changes, covering both the core application logic and the Hono web server's API endpoints. This allowed for confident refactoring and feature additions.
• Pexels API: Integrated for finding relevant Creative Commons licensed images, though the challenge of finding truly context-specific images remains an area for further refinement.

 % gemini 

 ███            █████████  ██████████ ██████   ██████ █████ ██████   █████ █████
░░░███         ███░░░░░███░░███░░░░░█░░██████ ██████ ░░███ ░░██████ ░░███ ░░███
  ░░░███      ███     ░░░  ░███  █ ░  ░███░█████░███  ░███  ░███░███ ░███  ░███
    ░░░███   ░███          ░██████    ░███░░███ ░███  ░███  ░███░░███░███  ░███
     ███░    ░███    █████ ░███░░█    ░███ ░░░  ░███  ░███  ░███ ░░██████  ░███
   ███░      ░░███  ░░███  ░███ ░   █ ░███      ░███  ░███  ░███  ░░█████  ░███
 ███░         ░░█████████  ██████████ █████     █████ █████ █████  ░░█████ █████
░░░            ░░░░░░░░░  ░░░░░░░░░░ ░░░░░     ░░░░░ ░░░░░ ░░░░░    ░░░░░ ░░░░░

Navigating the Refactoring Process

Refactoring, the art of rethinking one’s code architecture, is where ambition meets discipline. This process necessitates a shift towards a modular approach, an architectural goal as noble as it is tedious. The ultimate aspiration? Crafting a Node.js application where each module is its own unit, testable in isolation, and easily replaceable without a tectonic shift in the entire system. Fortunately this is the kind of task that AI can achieve without hype.

Heres the first prompt I tried. I asked Gemini CLI to make a proposal for refctoring and explain how it would do it.

> the app in this folder could use refactoring - it doesnt use direnv but should, and it only takes a simple parameter whereas i would prefer it to take a title and keywords to use. Lastly I'd like it to find a suitable image with a creative commons license. Explain how you would propose to refactor it and write your reasonings in a REFACTOR.md file, and explain how you would update the refactored app with the proposed functionality in CHANGES.md - I may get you to implement the changes but would like to see the proposal first

✦ Okay, I will analyze the codebase, then create REFACTOR.md to document the proposed refactoring and CHANGES.md to
  detail the new functionality.

The output was thougthful and complete. Heres an extract of it.

Ive reviewed the code. Now I will create the REFACTOR.md and CHANGES.md files.
 
✔  WriteFile Writing to REFACTOR.md                                                                                                                                                                                       
1
2 # Refactoring Plan                                                                                   
3                                                                                                        
4 This document outlines the proposed refactoring of the application.                          5        
6 ## Current Architecture                                                                                 7              
8 The application is a Node.js-based tool. Its key components are:                    9 
10 - **`index.js`**: The main entry point. It includes a simple web server to trigger content generation via a GET request.            
...             
13 - **`api.js`**: Manages interactions with external APIs, specifically OpenAI and Joomla.   
14 - **`logger.js`**: A simple `pino` logger.   
15 - **`package.json`**: Defines the projects dependencies and scripts.     
16                                    
17 ## Issues Identified      
18                
19 1.  **Environment Variable Management**: The application uses the `dotenv` package to load  environment variables from a `.env` file. This is a common practice, but it can be improved by using a tool like `direnv` to automatically load and unload environment variables when entering and leaving the project directory. This avoids the need to explicitly call `require('dotenv').config()` in the code.                                                                                                  20 2.  **Parameter Handling**: The application currently takes a single keyword as a parameter. This is limiting. It should be able to accept a more descriptive title and a list of keywords to improve the quality and relevance of the content.             
21 3.  **Lack of Image Integration**: The generated articles are text-only. To be more engaging, they should include a relevant image with a Creative Commons license.                                 
22 4.  **Monolithic `index.js`**: The `index.js` file has a lot of responsibilities, including argument parsing, orchestration, and serving the web interface. This could be broken down into smaller, more focused modules.                                                                                       
23 5.  **No Input Validation**: The application does not perform any validation on the input keyword,  which could lead to unexpected behavior.                                                               
24 6.  **Limited Error Handling**: While there is some basic error handling, it could be more robust to handle cases like API failures or network issues more gracefully.                                      
25 7.  **Hardcoded Values**: There are several hardcoded values in the code. These should be configurable.                     
26                                                                                                      
27 ## Proposed Refactoring                                                                                28                                                                                                       
29 1.  **Introduce `direnv`**: Replace the `dotenv` package with `direnv` for environment variable management. This will require creating a `.envrc` file and removing the `require('dotenv').config()` call from `index.js`.                                                                                
30 2.  **Improve Parameter Handling**: Modify the application to accept an article title and a list of keywords as command-line arguments or as parameters in the web interface. This will likely involve using a command-line argument parsing library like `yargs` or `commander`.                             31 3.  **Separate Concerns**: Break down the `index.js` file into smaller modules. For example, the web server could be moved to a separate `server.js` file, and the main application logic could be in a  `app.js` file.                                                                                         ...

I allowed it to implement the changes

please implement the proposed changes

✦ Okay, I will now implement the proposed changes. Here is my plan:
   1. Set up the environment: Ill create a .envrc file for direnv and add .env to the .gitignore file.
   2. Restructure the application: Ill split index.js into app.js (for the core logic) and server.js (for the Hono web
      server), and update index.js to be a command-line interface.
   3. Enhance functionality: Ill add the ability to specify a title and keywords, and integrate a Creative Commons image
      search using the Pexels API.
   4. Install dependencies: Ill add yargs for command-line argument parsing and pexels for image searching.

Watching for mistakes

Gemini CLI asks (by default) for confirmation of each step which allows you to modify what is being suggested. Here what Gemini CLI was suggesting would overwrite some exising data, I noticed and adjusted for it.

Now I will rename .env.example to .env and add a placeholder for the Pexels API key.
 ╭────────────────────────────────╮
 │ -  Shell mv .env.example .env  │
 ╰────────────────────────────────╯

> moving .env.example to .env would overwrite the data there

✦ You are right. I will not move the file. I will check if a .env file exists, and if not, I will copy .env.example to
  .env. Then I will add the PEXELS_API_KEY to the .env file.

Completing the refactoring

With some more iterations (with permission each time) editing the source code Gemini CLI announced it was finished.

I have completed the refactoring. Here is a summary of the changes:
   * Environment Variables: The application now uses direnv for environment variable management. A .envrc file has been created, and the .env file has been added to .gitignore.
   * Application Structure: The application has been restructured into app.js (core logic), server.js (web server), and index.js (command-line interface).
   * New Functionality:
       * The application now accepts an article title and keywords as input.
       * It can find and insert a Creative Commons licensed image from Pexels into the generated article.
   * Dependencies: The yargs and pexels packages have been added to package.json.
   * Scripts: The start script in package.json now runs the web server, and a new cli script has been added to run the
     command-line interface.

  You can now run the application in two ways:

   1. Web Server: npm start
   2. Command-Line Interface: npm run cli -- --title "Your Title" --keywords "keyword1, keyword2"

Generating the README.md

I wondered if Gemini CLI could do a better more logical job of the README.md. Turned out it could!

 ive partly updated README.md please finish

✦ Of course. Please provide me with the current content of README.md, and I will complete it for you. Or, I can read the file myself if you prefer.

> it is in the current folder

✦ Got it. Ill read the README.md file, complete it with the details of the recent refactoring, and then write the updated version back.
 ╭────────────────────────╮
 │ ✔  ReadFile README.md  │
 ╰────────────────────────╯
✦ Ive read the README.md file. I will now update it to reflect the recent changes, including instructions for setting up the environment and using the new features.

I have updated the README.md file with the latest information. It now includes instructions for setting up the environment and using the new command-line interface and server.

Iterating on the changes to make my environment variables secure but also documented 

there were still a few configuration issues with the app and its use of environment variables. Gemini CLI to the rescue again.

adjust the app to only use .envrc and not .env for simplicity, add the correct export statement before the variable in .envrc, and adjust the README.md to match

✦ Understood. I will simplify the environment variable management to use only a .envrc file.

I have updated the application to use only .envrc for environment variables and updated the README.md accordingly. The .env file has been removed.

Direnv envrc override

This next step, of using a local envrc file really is a great idea and although its obvious I dont think I would have come up with it at all. It works beautifully. It means that my GitHub repo has a fully documented envrc file and my local copy has credentials in envrc.local which work. Its a form of local override which I should have thoght about given Joomla works this way often.

the .envrc doesnt contain any credentials which is right for git, but i would like to run it - whats the best approach to keeping the .envrc this way for a clean check in but also running the tests I want to with valid credentials

✦ That's an excellent question and a common best practice. The best approach is to use a second, git-ignored file that contains your local overrides and secrets. direnv can be configured to load this file if it exists.

  Here's the plan:
   1. Create a local environment file: We'll use .envrc.local.
   2. Ignore the local file: Add .envrc.local to .gitignore to ensure it is never committed.
   3. Update `.envrc`: Modify it to source .envrc.local if it exists. This keeps the main .envrc clean for check-in while allowing local overrides.
   4. Update `README.md`: Adjust the setup instructions to reflect this new, more secure workflow.

  I'll start by updating the .gitignore file.
 ╭───────────────────────────────────────────╮
 │ ✔  Shell echo .envrc.local >> .gitignore  │
 ╰───────────────────────────────────────────╯
✦ Now, I'll modify the .envrc to source the local file.

8    +     
9    + if [ -f .envrc.local ]; then    
10   +   source .envrc.local 
11   + fi         
12   +

It took anther iteration, to move a hardcoded variable into the envrc.

ive modified .envrc and .envrc.local to take account of API_URL because I noticed that it was not working as expected. please adjust the code to take account of this

✦ Got it. I will update the code to use the API_URL environment variable.

And lastly updated the documentation

that seems to be working well please review the README.md, CHANGES.MD and REFACTOR.md and make one final README.md removing the other two files

✦ Of course. I will consolidate the information from README.md, CHANGES.MD, and REFACTOR.md into a final, comprehensive README.md, and then remove the other two files.

Committing the changes to GitHub using Gemini CLI

I use GitHub, and this folder is a repository, so heres another place where Gemini CLI can take the dridgery out of Git commands and make for a better process. I had to iterate this a couple of times to get the commit message the way I wanted but it got to a good point and works well.

thanks, now please commit all the changes to the git repository with a short relevant comment about the refactor following best practice for comments

✦ Of course. I'll now commit the changes with a descriptive message.

Adding automated Unit Testing for 100% test coverage, automatically with Gemini CLI

thats really great. now Id like to add some automated testing using a best practice framework for node and Javascript. Please make a proposal in TESTS.md but dont actually create the tests yet

✦ Excellent idea. Adding automated testing is a crucial step to ensure the application's reliability and maintainability.

 Gemini CLI created a proposal for a texint stragegy. Here it is, well part of it anyway. Its pretty good.

After a few iterations to fix and integrate the testing framework which took about 30 minutes the tests worked.

Excellent! All tests are now passing. We have successfully:

   * Installed and configured Jest.
   * Created a unit test for the core application logic, mocking external dependencies.
   * Created an integration test for the Hono server using hono/testing.
   * Fixed a bug in the server's error handling.

  The project now has a solid foundation for automated testing.

  I will now commit these changes to the repository.
 ╭──────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
 │ ✔  Shell git add . && git commit -m "feat: Add Jest testing framework   

Our testing strategy involved both unit and integration tests:

• Unit Tests: For core logic (e.g., in `app.js`), we mocked all external dependencies (API calls, file system operations) to ensure isolated and fast tests. This allowed us to verify the application's internal flow and logic with confidence.
• Integration Tests: For the Hono web server, we utilized `hono/testing` to directly interact with the application's routes. This ensured that our API endpoints were correctly handling requests and returning the expected responses, without the overhead of spinning up a full HTTP server.

This comprehensive testing approach allowed us to iterate rapidly on the content generation logic and refactor the codebase with confidence, knowing that any regressions would be quickly caught.

Iterative Refinement: The Human Element in AI Development

Our journey with this application was one of iterative refinement. AI outputs contained errors. It was through continuous human oversight, analysis of the AI's responses, and precise adjustments to the prompts that we achieved the desired quality.

The Feedback Loop: From Observation to Prompt Adjustment

This process involved a constant feedback loop:

1. Propose: Run the content generation, in the form of a proposal, analyze the output for, accuracy in meeting the requirement.
2. Identify Issues: Pinpoint specific areas where the AI fell short 
3. Adjust Proposal: Modify the system or user prompts to address the identified issues. This often involved adding more explicit constraints or refining the requirements.
4. Implement: Allow Gemini CLI to complete the work and evaluate the new output.

This hands-on approach, guided by a clear vision for the final product, is what truly elevates AI-powered application development beyond a novelty.

Concluding Thoughts: A Practical Approach to AI Development

In conclusion, refactoring a Node.js app more than just a technical exercise; it was a practical lesson in the nuances of AI development. It underscored that true innovation often lies not in chasing the latest AI model, but in the meticulous application of prompt engineering, robust architectural design, and comprehensive testing.

With thoughtful consideration and iterative refinement, AI can indeed be a powerful ally in crafting high-quality, apps. 

This is quite the result. It would have possibly taken me several long days work to do manually. Gemini CLI helped me choose the tooling to use, set up the testing strategy and build the tests with mocks following best practice. This is profound.