The Metadata Agent

The Metadata Agent analyzes your questionnaire metadata using artificial intelligence (AI) and suggests improvements to question labels, question types, answer groupings, filters, and other reporting metadata. It is built around pre-defined tasks that mirror actions already available on the Questions page. The agent relies exclusively on your project's existing metadata and does not access or process participant response data.

If AI features are disabled for your account, you cannot create a new analysis until AI features are enabled. Changes made based on previous analyses will still be viewable.

1: What is Metadata?

Metadata is the information that describes your questionnaire and reporting configuration rather than participant responses. In Forsta Visualizations, metadata includes all project information except for raw numerical or textual data. This includes question and variable names, labels, question types, response options, filters, and other reporting settings.

2: Using the Metadata Agent

1. While in Administrate, open your project. Select Variables from the left side bar. Then select the Questions tab.

2. On the right side of the Questions page, select Metadata Agent.

   

3. The Metadata agent panel displays on the right side of the page.

   

4. Select + SELECT TASKS, then select one or more tasks from the side panel for the metadata agent to complete. You can also use Select all or Deselect all to quickly manage your task selections. Tasks are organized by categories. You must select at least one analysis task to start an analysis. Once you have selected at least one task type, you can add custom rules for that task.

   

   See 2.1: Available Tasks and 2.2: Custom Rules for details on pre-designed tasks and creating custom rules.

5. Select START ANALYSIS.

   

6. Once the analysis has completed, review the generated list of suggested changes. Suggestions are grouped by category, making them easier to review before deciding which changes to apply. You can hover over the information icon “(i)” to view the explanation for a recommendation.

   

   If you are not satisfied with the generated suggestions, you can re-run the analysis. Forsta recommends you add or modify custom rules before re-running the analysis.

7. Approve all suggestions or only the changes you want to apply. Approved changes are highlighted immediately on the Questions page. You can approve suggestions from multiple analyses within the same session.

   You can use Quick filters to view unsaved modified data (whether modified manually or using the Metadata Agent).

8. Once you have made and/or approved all desired changes, select SAVE NOW (in the Metadata Agent) or Save on the Questions page. Selecting Save or Save now will save any changes made, whether modified manually or using the Metadata Agent.

   Selecting Continue without saving keeps your approved (unsaved) changes while allowing you to continue refining the analysis. You can modify tasks, update Custom Rules, run additional analyses, and combine approved changes from multiple analyses before saving everything to the project in a single operation.

   

You can refine your analysis at any time. If the results do not meet your expectations, you can:

• Edit your custom rules and run the analysis again.

• Add or remove analysis tasks.

• Run additional analysis without starting a new Metadata Agent session.

• Review and compare suggestions from multiple analyses before deciding which changes to apply.

This iterative workflow makes it easy to refine one task at a time. For example, you can focus on Question Labels until you are satisfied with the suggestions, then continue with Question Types or other tasks. Approved changes from multiple analyses remain available throughout the session until you save them.

To begin a completely separate analysis, select the New Analysis (+) icon in the upper-right corner of the Metadata Agent panel.

2.1: Available Tasks

The following tasks are available:

• Questions

  • Question Labels: Improve clarity and consistency of question labels

  • Question Tags: Suggest question tags to improve metadata organization

  • Question Colors: Recommend question color updates for better organization

  • Question Sorting: Recommend improved question ordering

  • Question Blocks: Suggest improved question block assignments

  • Question Types: Optimize question types for better data analysis

• Answers

  • Answer Labels: Enhance answer labels for improved readability

  • Answer Tags: Suggest answer tags to improve metadata organization

  • Answer Colors: Recommend answer color updates for clearer interpretation

  • Answer Sorting: Recommend improved answer ordering

  • Answer Groupings: Group similar answers for better analysis

• Filters

  • Recommended Filters: Get suggestions for the most effective filters to use in your analysis

• Cleanup

  • Delete Suggestions: Highlight questions that may be safe to remove

2.2: Custom Rules

Custom Rules replace the Metadata Agent's default rules for every selected task. The selected tasks are still performed, but suggestions are generated according to your Custom Rules instead of the built-in defaults. You must select at least one task in order to add custom rules. You can manually input custom instructions, or you can use the AI-assist to generate rules. Rules are integrated into the analysis request and can be modified directly from the results page to refine the output.

When you select Write Rules, a dialog opens where you can enter your instructions.

The custom rules should be instructions that guide how the Metadata Agent analyzes your metadata. For example, you can define a rule such as “Only change question type from Single Choice to Scale if the question contains exactly 10 answers.”. This helps ensure that suggested changes align more closely with your organization’s preferred metadata setup.

If you select Have AI generate rules, you can prompt AI to generate custom rules by describing how the analysis should behave.

The Metadata Agent generates a starting set of rules based on the selected tasks. Review the generated rules and modify them as needed before selecting Apply Rules.

After you save your rules, they appear below the selected analysis tasks in the Metadata Agent side panel. This lets you verify the instructions that will be included in the analysis request before you start.

If you are not satisfied with the generated suggestions, you can modify the custom rules directly from the analysis results and rerun the Metadata Agent. You can also adjust the selected tasks and update the custom rules before starting a new analysis, making it easier to refine the output until it matches your expectations.

2.2.1: Custom Rules Best Practice

When using Custom Rules with multiple tasks, clearly specify which task(s) each rule should apply to. If different tasks require different rules, consider running separate analyses. This provides greater control over the generated suggestions and makes the results easier to review.

For example, if you are updating Question Labels and Question Colors in the same analysis, your Custom Rules could look like this:

Question Labels

• Convert all labels to sentence case.

• Do not modify demographic questions.

Question Colors

• Color NPS questions green (#008000).

• Color demographic questions blue (#0000FF).

• Leave all other question colors unchanged.

3: Metadata Agent change logs

Select the Logs icon to view changes implemented based on previous Metadata analysis suggestions.

Manual changes are not recorded in this log. If you manually edit a label that was also suggested by the Metadata Agent, your manual change will not be recorded in the Metadata change log.

If AI features are disabled for your account, you can still view previous analyses. However, you cannot create a new analysis until AI features are (re)enabled.

4: Data Security

As always, your data privacy and security are our top priorities. Here’s how your data is handled with Azure OpenAI:

Your metadata (inputs) and AI-generated suggestions (outputs):

• Are NOT available to other customers.

• Are NOT available to OpenAI.

• Are NOT used to improve OpenAI models.

• Are NOT used to improve any Microsoft or third-party products or services.

• Are NOT used for automatically improving Azure OpenAI models. The models are stateless and do not retain data from your interactions.

The Azure OpenAI Service is fully controlled by Microsoft. Microsoft hosts the OpenAI models in their Azure environment, and the service does NOT interact with any services operated by OpenAI (e.g., ChatGPT or the OpenAI API).

Your data will be kept in the same region as your server.

4.1: Reliability of Large Language Models

Large language models like Azure OpenAI are powerful but not perfect. Here are some key points:

• Context Understanding: The AI does well with context but can sometimes misinterpret unclear or complex inputs.

• No Learning from Use: The AI doesn’t learn from your input, ensuring your data stays private.

• Human Review: Always review AI-generated outputs to ensure they meet your needs and expectations.