Scribd
Upload
18 views3 pages

Generative AI for Python Code Analysis

The document discusses the use of Generative AI in automating tasks throughout the Software Development Life Cycle (SDLC), particularly in understanding Python code during development and testing phases. It outlines a solution approach that includes repository cloning, file parsing, chunking strategies, embedding generation, and querying with language models to facilitate code comprehension. Additionally, it highlights enhancements for retrieval processes and provides a use case example demonstrating how the system can answer specific queries about code functionalities.

Uploaded by

vijayantp
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
18 views3 pages

Generative AI for Python Code Analysis

The document discusses the use of Generative AI in automating tasks throughout the Software Development Life Cycle (SDLC), particularly in understanding Python code during development and testing phases. It outlines a solution approach that includes repository cloning, file parsing, chunking strategies, embedding generation, and querying with language models to facilitate code comprehension. Additionally, it highlights enhancements for retrieval processes and provides a use case example demonstrating how the system can answer specific queries about code functionalities.

Uploaded by

vijayantp
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd

Source Code Analysis using Generative AI

Problem Statement & Usage:


Gen AI can be used in various SDLC stages to automate various tasks in requirement analysis, design,
development, testing phase.
In Development and testing phase the project team members, testers need to understand the code
and functionalities quickly to get overview of the project. The new project members also need to get
used to the code and project.
We can ingest any python-based repository project consisting of modules, py files and perform QA
on the code asking the functionality /purpose/usage of a particular class, function, code module and
what does it do and what does it contain.

Solution Approach

Main Components:

Repository Cloning: Automatically duplicate the target repository for examination.

File Loading and Parsing: Retrieve and analyze Python files from the repository.

Chunking: Divide the code into smaller, manageable sections while maintaining context.

Embedding Generation: Generate vector representations of code snippets using embedding


models.

Knowledge Base Creation: Store these embeddings in a vector database for fast retrieval.

Querying with LLM: Utilize a language model to answer specific queries about the code.

Chunking Strategy

We have different chunking strategies and techniques depending upon type of data and problem
statement and retrieval complexity.

• Character Splitting - Simple static character chunks of data

• Recursive Character Text Splitting - Recursive chunking based on a list of separators.

• Document Specific Splitting - Various chunking methods for different document types (PDF, Python, Markdown)

• Semantic Splitting - Embedding walk based chunking

• Agentic Splitting - Agentic chunker using propositions based .

Example: Vijayant went to the ground. He likes walking > [ Vijayant went to the ground.', Vijayant likes walking']
Context Aware Chunking:
We have used context aware splitting here which will take care of chunking based on code structure.
It will extract the code inside a python function and keep track of code chunks belonging to particular
function. It will extract various chunks and label them under a particular function or a class.

Embeddings:

We have used Open AI GPT 4 API to generate the dense embeddings and store them in chroma
Vector Db.

We have used also memory in the QA chats by summarizing the history chat and adding that to the
current question being asked. In langchain we have used conversation summary memory.

Retrieval Process ( Pre & Post)

Pre-retrieval optimizations: Before starting the search, the system refines the query for
better results. For example, Query Transformations and Query Routing

Example: A query like "latest advances in AI" could be transformed into "recent
breakthroughs in artificial intelligence" to better target relevant documents.

Example: If the query is about scientific research, routing could direct it to a specialized
database of research papers rather than a general knowledge base.

Enhanced retrieval techniques: During the search, Hybrid Search integrates both keyword
and semantic searches, ensuring a thorough scan. By using Chunking and Vectorization, large
documents are broken into smaller sections that are vectorized.

Post-retrieval refinements: After gathering the information, Reranking and Filtering steps
assess the relevance of the retrieved chunks. Rather than simply selecting the top results,
these processes evaluate the most useful data

We have used MMR (Maximal Marginal Relevance) for retrieval to improve diversity and relevance
of search result. We also have top k to retrieve the top k most similar docs based on cosine
similarity.

For RAG retrieval we have used Langchain conversational retrieval chain pipeline which will take care
of internal prompting and querying.

Use case Example Use Case:

In this example, the system is used to query a codebase to understand specific parts, such as a
machine learning training pipeline. By asking a question like "What is happening in the training
pipeline?", the language model retrieves relevant code snippets and provides a detailed explanation
of the process. The code demonstrates this interaction by running the query and printing the
explanation
question = "what is happening in training pipeline?"

result = qa(question)

print(result['answer'])

question = "what is data ingestion class and what does it do ?"

result = qa(question)

print(result['answer'])

The purpose of the data ingestion class is to ingest data from the source formats , It is located in
components module .This includes the downloading of file from the passed URL and and extracting a
zip into data directory given.

Possible Enhancements
1. Add advanced RAG methods like query transformation and routing methods and post
retrieval by adding more context.
2. Also devise strategy to chunk the code doc strings and comments and store them and create
summary of the doc strings and create embeddings and store in vectorDB.
3. Add metadata also in indexing

Common questions

Powered by AI

Conversation summary memory is utilized in Langchain during QA chats by summarizing the history of the chat and appending it to the current question being asked. This approach ensures that the context of the conversation is maintained, allowing for more coherent and relevant responses from the system .

Incorporating advanced RAG methods like query transformation and routing can significantly improve retrieval accuracy by tailoring queries to more precise forms and directing them to the most appropriate sources. This targeted approach enhances the ability to extract detailed and specific information from complex codebases efficiently .

Querying a codebase to understand a machine learning training pipeline highlights the process whereby specific parts of the code can be identified and explained through detailed querying. This interaction, facilitated by language models, helps users comprehend complex processes by retrieving and elucidating relevant code snippets, such as detailing what happens in the 'training pipeline', demonstrating the pipeline's logic and steps .

Potential enhancements include devising strategies to chunk code docstrings and comments, summarize them, create embeddings, and store these in the vector database. Additionally, adding metadata during the indexing process could further improve retrieval efficiency and contextual understanding .

Pre-retrieval optimizations improve the efficacy by refining the query through transformations and routing to more relevant sources. This ensures that queries are better targeted, such as changing 'latest advances in AI' into 'recent breakthroughs in artificial intelligence', or routing scientific queries to specialized databases instead of general ones, thus enhancing the relevance of retrieval results .

Embeddings play a critical role in converting code snippets into dense vector representations using models like OpenAI's GPT-4 API. These vectors are stored in a vector database such as chroma, facilitating fast and efficient retrieval of code information by allowing similarity comparisons for relevant code analysis and understanding .

Post-retrieval refinements such as reranking and filtering enhance the usefulness by assessing the relevance of retrieved chunks and highlighting the most valuable data. This process ensures that the system doesn't just select top results based on a superficial score but evaluates the chunks' actual utility and pertinence .

Hybrid search integrates both keyword and semantic searches by performing a thorough scan of documents using chunking and vectorization. This allows for a comprehensive retrieval of relevant data, where keyword search identifies surface-level matches while semantic search delves deeper into the context of the content .

Context-aware chunking enhances the understanding of a codebase by extracting code inside a Python function and maintaining context, allowing clear segregation and labeling of code chunks under particular functions or classes. This helps developers and testers to quickly assimilate the code structure and logic, facilitating easier navigation and understanding during development and testing phases .

Maximal Marginal Relevance (MMR) is significant as it balances relevance and diversity in search results, ensuring that the retrieved information is not only pertinent but also varied, thereby preventing redundancy and encouraging comprehensive understanding of the subject matter .

You might also like