Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Update dspapi_build.yaml for code coverage report #105

Open
wants to merge 21 commits into
base: PSL-DEV
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
4274f78
create a function to validate the required fields
pradeepjha-microsoft Nov 28, 2024
8bc5dbc
Revert the changes
pradeepjha-microsoft Nov 28, 2024
11b7f39
Implemented the required validation
pradeepjha-microsoft Nov 28, 2024
f6e4ce2
Implemented the validation with failure banner.
pradeepjha-microsoft Dec 2, 2024
766ff86
refactor to allowKeywordExtractingHandler to read from prompt txt
vtjc2002 Dec 4, 2024
8d2d66c
Added a common function to validate the required variables
pradeepjha-microsoft Dec 5, 2024
bb89f62
Added the correct variable name
pradeepjha-microsoft Dec 5, 2024
ea42bdd
Merge pull request #21 from microsoft/psl-dkm-us-11110
pradeepjha-microsoft Dec 5, 2024
928971b
remove $fqdn= $null
pradeepjha-microsoft Dec 6, 2024
c0040ed
Merge pull request #24 from microsoft/psl-dkm-us-11110
pradeepjha-microsoft Dec 6, 2024
20b93ab
Added CodeOwners, PRTemplate and StaleBot
Prasanjeet-Microsoft Dec 10, 2024
f8ca000
Updated CodeOwners
Prasanjeet-Microsoft Dec 11, 2024
72e1cbb
Added ISSUE_TEMPLATE
Prasanjeet-Microsoft Dec 13, 2024
929229f
Merge pull request #25 from Prasanjeet-Microsoft/Adding_CodeOwners_PR…
Roopan-Microsoft Dec 13, 2024
d7dc007
Merge pull request #23 from vtjc2002/main
Dongbumlee Dec 13, 2024
d13581a
Align extract-keywords.txt Prompt with Hardcoded System Message
Prasanjeet-Microsoft Dec 16, 2024
1c97936
Merge pull request #101 from microsoft/PSL-UpdatePrompt
Roopan-Microsoft Dec 17, 2024
e49ed7e
Added PR title checker GitHub Actions workflow
Prasanjeet-Microsoft Dec 18, 2024
20b5894
Merge pull request #102 from Prasanjeet-Microsoft/Add_pr-title-checker
Prasanjeet-Microsoft Dec 20, 2024
9b9c8e8
Update dspapi_build.yaml for code coverage report
pradeepjha-microsoft Dec 24, 2024
de60329
Remove TestAndCoverage stage from pipeline
pradeepjha-microsoft Dec 27, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .github/CODEOWNERS
Validating CODEOWNERS rules …
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# Lines starting with '#' are comments.
# Each line is a file pattern followed by one or more owners.

# These owners will be the default owners for everything in the repo.
* @Avijit-Microsoft @Roopan-Microsoft @Prajwal-Microsoft @dongbumlee
45 changes: 45 additions & 0 deletions .github/ISSUE_TEMPLATE/bug_report.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
---
name: Bug report
about: Create a report to help us improve
title: ''
labels: bug
assignees: ''

---

# Describe the bug
A clear and concise description of what the bug is.

# Expected behavior
A clear and concise description of what you expected to happen.

# How does this bug make you feel?
_Share a gif from [giphy](https://giphy.com/) to tells us how you'd feel_

---

# Debugging information

## Steps to reproduce
Steps to reproduce the behavior:
1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error

## Screenshots
If applicable, add screenshots to help explain your problem.

## Logs

If applicable, add logs to help the engineer debug the problem.

---

# Tasks

_To be filled in by the engineer picking up the issue_

- [ ] Task 1
- [ ] Task 2
- [ ] ...
32 changes: 32 additions & 0 deletions .github/ISSUE_TEMPLATE/feature_request.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
---
name: Feature request
about: Suggest an idea for this project
title: ''
labels: enhancement
assignees: ''

---

# Motivation

A clear and concise description of why this feature would be useful and the value it would bring.
Explain any alternatives considered and why they are not sufficient.

# How would you feel if this feature request was implemented?

_Share a gif from [giphy](https://giphy.com/) to tells us how you'd feel. Format: ![alt_text](https://media.giphy.com/media/xxx/giphy.gif)_

# Requirements

A list of requirements to consider this feature delivered
- Requirement 1
- Requirement 2
- ...

# Tasks

_To be filled in by the engineer picking up the issue_

- [ ] Task 1
- [ ] Task 2
- [ ] ...
22 changes: 22 additions & 0 deletions .github/ISSUE_TEMPLATE/subtask.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
---
name: Sub task
about: A sub task
title: ''
labels: subtask
assignees: ''

---

Required by <link to parent issue>

# Description

A clear and concise description of what this subtask is.

# Tasks

_To be filled in by the engineer picking up the subtask

- [ ] Task 1
- [ ] Task 2
- [ ] ...
37 changes: 37 additions & 0 deletions .github/PULL_REQUEST_TEMPLATE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
## Purpose
<!-- Describe the intention of the changes being proposed. What problem does it solve or functionality does it add? -->
* ...

## Does this introduce a breaking change?
<!-- Mark one with an "x". -->

- [ ] Yes
- [ ] No

<!-- Please prefix your PR title with one of the following:
* `feat`: A new feature
* `fix`: A bug fix
* `docs`: Documentation only changes
* `style`: Changes that do not affect the meaning of the code (white-space, formatting, missing semi-colons, etc)
* `refactor`: A code change that neither fixes a bug nor adds a feature
* `perf`: A code change that improves performance
* `test`: Adding missing tests or correcting existing tests
* `build`: Changes that affect the build system or external dependencies (example scopes: gulp, broccoli, npm)
* `ci`: Changes to our CI configuration files and scripts (example scopes: Travis, Circle, BrowserStack, SauceLabs)
* `chore`: Other changes that don't modify src or test files
* `revert`: Reverts a previous commit
* !: A breaking change is indicated with a `!` after the listed prefixes above, e.g. `feat!`, `fix!`, `refactor!`, etc.
-->

## Golden Path Validation
- [ ] I have tested the primary workflows (the "golden path") to ensure they function correctly without errors.

## Deployment Validation
- [ ] I have validated the deployment process successfully and all services are running as expected with this change.

## What to Check
Verify that the following are valid
* ...

## Other Information
<!-- Add any other helpful information that may be needed here. -->
22 changes: 22 additions & 0 deletions .github/workflows/pr-title-checker.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
name: "pr-title-checker"

on:
pull_request_target:
types:
- opened
- edited
- synchronize
merge_group:

permissions:
pull-requests: read

jobs:
main:
name: Validate PR title
runs-on: ubuntu-latest
if: ${{ github.event_name != 'merge_group' }}
steps:
- uses: amannn/action-semantic-pull-request@v5
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
19 changes: 19 additions & 0 deletions .github/workflows/stale-bot.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
name: 'Close stale issues and PRs'
on:
schedule:
- cron: '30 1 * * *'

permissions:
contents: write
issues: write
pull-requests: write

jobs:
stale:
runs-on: ubuntu-latest
steps:
- uses: actions/stale@v9
with:
stale-issue-message: 'This issue is stale because it has been open 180 days with no activity. Remove stale label or comment or this will be closed in 30 days.'
days-before-stale: 180
days-before-close: 30
1 change: 1 addition & 0 deletions App/kernel-memory/service/Abstractions/Constants.cs
Original file line number Diff line number Diff line change
Expand Up @@ -163,4 +163,5 @@ public static class Summary
// Standard prompt names
public const string PromptNamesSummarize = "summarize";
public const string PromptNamesAnswerWithFacts = "answer-with-facts";
public const string PromptNamesExtractKeywords = "extract-keywords";
}
1 change: 1 addition & 0 deletions App/kernel-memory/service/Core/Core.csproj
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,7 @@
<ItemGroup>
<EmbeddedResource Include="Prompts\summarize.txt" />
<EmbeddedResource Include="Prompts\answer-with-facts.txt" />
<EmbeddedResource Include="Prompts\extract-keywords.txt" />
</ItemGroup>

<ItemGroup>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@
using Microsoft.SemanticKernel;
using Microsoft.SemanticKernel.ChatCompletion;
using static Microsoft.KernelMemory.Pipeline.DataPipeline;
using Microsoft.KernelMemory.Prompts;

namespace Microsoft.KernelMemory.Handlers;

Expand All @@ -21,11 +22,13 @@ public sealed class KeywordExtractingHandler : IPipelineStepHandler
private readonly IPipelineOrchestrator _orchestrator;
private readonly Kernel _kernel;
private readonly KernelMemoryConfig? _config = null;
private readonly string _extractKeywordPrompt;

public KeywordExtractingHandler(
string stepName,
IPipelineOrchestrator orchestrator,
KernelMemoryConfig config = null,
IPromptProvider? promptProvider = null,
ILoggerFactory? loggerFactory = null
)
{
Expand All @@ -34,6 +37,10 @@ public KeywordExtractingHandler(
this._orchestrator = orchestrator;
this._config = config;

promptProvider ??= new EmbeddedPromptProvider();

this._extractKeywordPrompt = promptProvider.ReadPrompt(Constants.PromptNamesExtractKeywords);

//init Semantic Kernel
this._kernel = Kernel.CreateBuilder()
.AddAzureOpenAIChatCompletion(deploymentName: (string)this._config.Services["AzureOpenAIText"]["Deployment"],
Expand Down Expand Up @@ -79,30 +86,7 @@ public KeywordExtractingHandler(
var chat = this._kernel.GetRequiredService<IChatCompletionService>();
var chatHistory = new ChatHistory();

var systemMessage = """
You are an assistant to analyze Content and Extract Tags by Content.
[EXTRACT TAGS RULES]
IT SHOULD BE A LIST OF DICTIONARIES WITH CATEGORY AND TAGS
TAGS SHOULD BE CATEGORY SPECIFIC
TAGS SHOULD BE A LIST OF STRINGS
TAGS COUNT CAN BE UP TO 10 UNDER A CATEGORY
CATEGORY COUNT CAN BE UP TO 10
DON'T ADD ANY MARKDOWN EXPRESSION IN YOUR RESPONSE
[END RULES]

[EXAMPLE]
[
{
[category1": ["tag1", "tag2", "tag3"]
},
{
"category2": ["tag1", "tag2", "tag3"]
}
]
[END EXAMPLE]
""";

chatHistory.AddSystemMessage(systemMessage);
chatHistory.AddSystemMessage(this._extractKeywordPrompt);
chatHistory.AddUserMessage($"Extract tags from this content : {extactedFileContent} \n The format should be Json but Markdown expression.");

var executionParam = new PromptExecutionSettings()
Expand Down
24 changes: 14 additions & 10 deletions App/kernel-memory/service/Core/Prompts/extract-keywords.txt
Original file line number Diff line number Diff line change
@@ -1,21 +1,25 @@
[EXTRACT KEYWORDS RULES]
IT SHOULD BE A LIST OF DICTIONARIES WITH CATEGORY AND KEYWORDS
KEYWORDS SHOULD BE CATEGORY SPECIFIC
KEYWORDS SHOULD BE A LIST OF STRINGS
KEYWORDS COUNT CAN BE UP TO 50
You are an assistant to analyze Content and Extract Tags by Content.
[EXTRACT TAGS RULES]
IT SHOULD BE A LIST OF DICTIONARIES WITH CATEGORY AND TAGS
TAGS SHOULD BE CATEGORY SPECIFIC
TAGS SHOULD BE A LIST OF STRINGS
TAGS COUNT CAN BE UP TO 10 UNDER A CATEGORY
CATEGORY COUNT CAN BE UP TO 10
DON'T ADD ANY MARKDOWN EXPRESSION IN YOUR RESPONSE
[END RULES]

[EXAMPLE]
[
{
"category1": ["keyword1", "keyword2", "keyword3"]
"category1": ["tag1", "tag2", "tag3"]
},
{
"category2": ["keyword1", "keyword2", "keyword3"]
"category2": ["tag1", "tag2", "tag3"]
}
]
[END EXAMPLE]

Extract Keywords from this :
{{$input}}
Extract Tags from this :
{{$input}}



Loading