Build scalable data solutions with SQL database in Microsoft Fabric

 

Exercise 1 - Setting up the Environment

Task-1: Create a New Fabric Workspace










Uploading: 246300 of 246300 bytes uploaded.

Task-2: Create a SQL Database in Microsoft Fabric





Task-3: Load the database with Sample data






Task-4: Working with the SQL database in Microsoft Fabric



Task-5: Exploring SQL analytics Endpoint












Exercise 2 - Copilot Capabilities for SQL database in Microsoft Fabric




--Create a query to get the product that is selling the most






Task-2: Copilot Quick Actions within the Query Window



SELECT c.CustomerID, c.FirstName,c.LastName,
COUNT(so.SalesOrderID) AS TotalPurchases,
SUM(so.SubTotal) AS TotalSpent,
AVG(so.SubTotal) AS AverageOrderValue,
MAX(so.OrderDate) AS LastPurchaseDate
FROM
SalesLT.Customer AS c JOIN SalesLT.SalesOrderHeader AS so ON c.CustomerID = so.CustomerID
GROUP BY c.CustomerID, c.FName, c.LName ORDER BY TotalSpent DESC;











Task-3: Using Copilot Chat Pane - Natural Language to SQL











Task-4: Chat Pane - Get results from Copilot





Task-5: Chat Pane - Write (with approval)








Exercise 3 - RAG Implementation with SQL Database in Fabric

Task-1: Create OpenAI resource in Azure Portal



















Task-2: Create Model Deployment for text-embedding-ada-002











Task-3: Setup of database credential



if not exists(SELECT * FROM sys.symmetric_keys WHERE [name] = '##MS_DatabaseMasterKey##')
begin
    CREATE master key encryption by password = N'V3RYStr0NGP@ssw0rd!';
end
go
if exists(SELECT * FROM sys.[database_scoped_credentials] where name = 
'Paste the OpenAI endpoint here')
begin
    DROP database scoped credential [Paste the OpenAI endpoint here];
end
CREATE database scoped credential [Paste the OpenAI endpoint here]
with identity = 'HTTPEndpointHeaders', secret = '{"api-key": "Paste the OpenAI Key here"}';
go


Task-4: Create embeddings for relational data


ALTER TABLE [SalesLT].[Product]
ADD  embeddings VECTOR(1536), chunk nvarchar(2000);



CREATE or ALTER PROCEDURE SalesLT.create_embeddings
(
    @input_text nvarchar(max),
    @embedding vector(1536) output
)
AS
BEGIN
DECLARE @url varchar(max) = '[Paste the OpenAI Endpoint here]openai/deployments/text-embedding-ada-002/embeddings?api-version=2024-06-01';
DECLARE @payload nvarchar(max) = json_object('input': @input_text);
DECLARE @response nvarchar(max);
DECLARE @retval int;

-- Call to Azure OpenAI to get the embedding of the search text
BEGIN try
    EXEC @retval = sp_invoke_external_rest_endpoint
        @url = @url,
        @method = 'POST',
        @credential = [Paste the OpenAI Endpoint here], 
        @payload = @payload,
        @response = @response output;
END try
BEGIN catch
    SELECT 
        'SQL' as error_source, 
        error_number() as error_code,
        error_message() as error_message
    return;
end catch
if (@retval != 0) begin
    SELECT 
        'OPENAI' as error_source, 
        json_value(@response, '$.result.error.code') as error_code,
        json_value(@response, '$.result.error.message') as error_message,
        @response as error_response
    return;
end
-- Parse the embedding returned by Azure OpenAI
DECLARE @json_embedding nvarchar(max) = json_query(@response, '$.result.data[0].embedding');

-- Convert the JSON array to a vector and set return parameter
set @embedding = CAST(@json_embedding AS VECTOR(1536));
END;




SET NOCOUNT ON
DROP TABLE IF EXISTS #MYTEMP 
DECLARE @ProductID int
DECLARE @text nvarchar(max);
DECLARE @vector vector(1536);
SELECT * INTO #MYTEMP FROM [SalesLT].Product
SELECT @ProductID = ProductID FROM #MYTEMP
SELECT TOP(1) @ProductID = ProductID FROM #MYTEMP
WHILE @@ROWCOUNT <> 0
BEGIN
    SET @text = (SELECT p.Name + ' '+ ISNULL(p.Color,'No Color') + ' '+  c.Name + ' '+  m.Name + ' '+  ISNULL(d.Description,'')
                    FROM 
                    [SalesLT].[ProductCategory] c,
                    [SalesLT].[ProductModel] m,
                    [SalesLT].[Product] p
                    LEFT OUTER JOIN
                    [SalesLT].[vProductAndDescription] d
                    on p.ProductID = d.ProductID
                    and d.Culture = 'en'
                    WHERE p.ProductCategoryID = c.ProductCategoryID
                    and p.ProductModelID = m.ProductModelID
                    and p.ProductID = @ProductID);
    exec SALESLT.create_embeddings @text, @vector output;
    UPDATE [SalesLT].[Product] SET [embeddings] = @vector, [chunk] = @text WHERE ProductID = @ProductID;
    DELETE FROM #MYTEMP WHERE ProductID = @ProductID
    SELECT TOP(1) @ProductID = ProductID FROM #MYTEMP
END



SELECT COUNT(*) FROM SalesLT.Product WHERE embeddings is null;













































































































































































































































Comments

Post a Comment

Popular posts from this blog

AI-900-3,4

Image
Module : 3 Get Started with Machine Learning in Azure Machine learning drives today’s AI by turning data into insights for predictions and recommendations. Creating an ML solution requires key decisions that affect cost, speed, and quality. Using Microsoft Azure, the process follows six main steps: Define the Problem: Decide what to predict and how to measure success. Get the Data: Find and access reliable data sources. Prepare the Data: Clean, explore, and format the data for modeling. Train the Model: Choose algorithms and adjust parameters. Integrate the Model: Deploy it to generate predictions. Monitor the Model: Track performance and retrain as needed. Machine learning is an ongoing process. Models evolve as data changes. Step 1: Define the Problem The first step in building a machine learning (ML) solution is to define the problem. You need to clearly understand what you want the model to predict and how you will measure success. Key Considerations Model output: What result shoul...
Read more

AI-900 12,13

Module : 12   Overview of Information Extraction * Information extraction is a process that uses AI to pull structured data from unstructured or semi-structured documents, like receipts, invoices, forms, and scanned files. It combines computer vision, OCR, and machine learning or generative AI. * Key Steps in an Information Extraction Pipeline 1. Text Detection & Extraction (OCR) - It uses computer vision to find text areas in an image. - The system extracts raw text from scanned or image-based documents. 2. Value Identification & Mapping It maps the OCR text to specific data fields using: - Machine learning - Rule-based mapping - Generative AI (in newer solutions) Example From a scanned receipt, the system extracts fields such as: - Vendor - Date - Subtotal - Tax - Total * Choosing the Right Approach When designing an information extraction solution, consider: Document Characteristics - Layout consistency - Fixed templates lead to simple, rule-based extraction - Many forma...
Read more

AI-900 10,11

Image
Module : 10  Introduction to Computer Vision Concepts  Computer Vision Tasks & Techniques   * What is Computer Vision? Computer vision uses AI techniques to process and understand visual input such as: - Images   - Videos   - Live camera streams   - It enables machines to "see" and interpret visual information.   1. Image Classification   - This predicts a single label for an entire image.   - The model is trained using many labeled images.   - For example, a smart grocery checkout can identify an apple, orange, or banana from a single item placed on a scale.   - Use case: Recognizing the main subject in an image.   2. Object Detection   - This identifies multiple objects in a single image.   It returns:   - Object labels   - Bounding box coordinates   - For example, a checkout system can detect all fruits placed together....
Read more