SQL Subqueries: A Practical Guide with Examples 📝

1. Find the Second-Highest Salary using a Subquery

To find the second-highest salary in the Employees table, we use a subquery with MAX():

sqlCopyEditSELECT MAX(Salary) 
FROM Employees 
WHERE Salary < (SELECT MAX(Salary) FROM Employees);

Explanation

• The inner query:

    sqlCopyEditSELECT MAX(Salary) FROM Employees;
  

  • Fetches the highest salary.

• The outer query:

    sqlCopyEditSELECT MAX(Salary) FROM Employees WHERE Salary < (subquery);
  

  • Finds the maximum salary less than the highest salary (i.e., the second-highest salary).

🔹 Example Table: Employees

👉 Output: 7000 (Second-Highest Salary)

---

2. Find Employees Earning Above Their Department's Average Salary

To get employees earning more than the average salary of their department, use:

sqlCopyEditSELECT EmployeeID, Name 
FROM Employees 
WHERE Salary > (
    SELECT AVG(Salary) 
    FROM Employees 
    WHERE DepartmentID = Employees.DepartmentID
);

How It Works

• The subquery:

    sqlCopyEditSELECT AVG(Salary) FROM Employees WHERE DepartmentID = Employees.DepartmentID;
  

  • Calculates the average salary per department.

• The outer query:

    sqlCopyEditSELECT EmployeeID, Name FROM Employees WHERE Salary > (subquery);
  

  • Fetches employees earning above their department’s average salary.

🔹 Example Table: Employees

👉 Output: Employees with ID 2 and 3 (Earning above their department’s average)

---

3. Retrieve All Departments Where No Employee Earns More than $50,000

To list departments where all employees earn $50,000 or less, use:

sqlCopyEditSELECT DepartmentID 
FROM Departments 
WHERE NOT EXISTS (
    SELECT 1 
    FROM Employees 
    WHERE Employees.DepartmentID = Departments.DepartmentID 
    AND Salary > 50000
);

Explanation

• The subquery:

    sqlCopyEditSELECT 1 FROM Employees WHERE Employees.DepartmentID = Departments.DepartmentID AND Salary > 50000;
  

  • Checks if any employee in a department earns above $50,000.

• The outer query:

    sqlCopyEditWHERE NOT EXISTS (subquery);
  

  • Filters departments where no employee earns more than $50,000.

🔹 Example Table: Employees

👉 Output: Departments 101 and 103 (because no employee earns above $50,000)

---

4. Find Customers Who Have Placed an Order (Subquery in WHERE Clause)

To get customers who have placed at least one order, use:

sqlCopyEditSELECT CustomerID, Name 
FROM Customers 
WHERE CustomerID IN (
    SELECT DISTINCT CustomerID FROM Orders
);

How It Works

• The subquery:

    sqlCopyEditSELECT DISTINCT CustomerID FROM Orders;
  

  • Gets the list of customers who have placed orders.

• The outer query:

    sqlCopyEditWHERE CustomerID IN (subquery);
  

  • Filters only customers present in the subquery results.

🔹 Example Table: Customers

🔹 Example Table: Orders

👉 Output: Customers Alice and Charlie (who placed orders)

---

5. Find Employees with the Same Job Title as ‘John’

To list employees having the same job title as ‘John’, use:

sqlCopyEditSELECT Name 
FROM Employees 
WHERE JobTitle = (
    SELECT JobTitle FROM Employees WHERE Name = 'John'
);

Explanation

• The subquery:

    sqlCopyEditSELECT JobTitle FROM Employees WHERE Name = 'John';
  

  • Fetches John’s job title.

• The outer query:

    sqlCopyEditWHERE JobTitle = (subquery);
  

  • Selects all employees with the same job title.

🔹 Example Table: Employees

👉 Output: Employees John and Mark (Both are Managers)

---

Final Thoughts

💡 Subqueries in SQL help solve complex queries efficiently by allowing nested queries inside SELECT, WHERE, and HAVING clauses.

🔥 Key Takeaways ✔️ Use MAX() with a subquery to find the second-highest salary.
✔️ Use AVG() in subqueries to compare employee salaries with department averages.
✔️ Use NOT EXISTS() to filter departments where no employees earn above a limit.
✔️ Use IN() with subqueries to match records across tables efficiently.
✔️ Use correlated subqueries when filtering based on conditions within the same table.

---

Mastering Correlated Subqueries in SQL with Examples 🚀

1. What is a Correlated Subquery?

   A correlated subquery is a subquery that references columns from the outer query. Unlike regular subqueries, it executes once per row of the outer query, making it useful for row-wise comparisons.

   ---

   1. Find Employees Earning More than the Average Salary in Their Department

   To fetch employees who earn more than their department’s average salary, use:

    sqlCopyEditSELECT EmployeeID, Name 
    FROM Employees e1 
    WHERE Salary > (
        SELECT AVG(Salary) 
        FROM Employees e2 
        WHERE e2.DepartmentID = e1.DepartmentID
    );
   

   How It Works

   • The subquery:

       sqlCopyEditSELECT AVG(Salary) FROM Employees e2 WHERE e2.DepartmentID = e1.DepartmentID;
     

     • Computes the average salary for each department.

   • The outer query:

       sqlCopyEditSELECT EmployeeID, Name FROM Employees WHERE Salary > (subquery);
     

     • Fetches employees earning above their department’s average salary.

🔹 Example Table: Employees

👉 Output: Employees Jane and Mark (Earning above their department’s average)

---

2. Retrieve the Most Recently Hired Employee in Each Department

To get the latest hired employee per department, use:

    sqlCopyEditSELECT EmployeeID, Name, DepartmentID 
    FROM Employees e1 
    WHERE HireDate = (
        SELECT MAX(HireDate) 
        FROM Employees e2 
        WHERE e2.DepartmentID = e1.DepartmentID
    );

Explanation

• The subquery:

    sqlCopyEditSELECT MAX(HireDate) FROM Employees e2 WHERE e2.DepartmentID = e1.DepartmentID;
  

  • Finds the latest hire date for each department.

• The outer query:

    sqlCopyEditSELECT EmployeeID, Name, DepartmentID FROM Employees WHERE HireDate = (subquery);
  

  • Selects employees with the latest hire date.

🔹 Example Table: Employees

👉 Output: Employees Jane and Mark (Most recently hired in their departments)

---

3. Find Customers Who Made the Most Recent Order

To fetch customers who placed the latest order, use:

    sqlCopyEditSELECT CustomerID, Name 
    FROM Customers c 
    WHERE OrderDate = (
        SELECT MAX(OrderDate) 
        FROM Orders o 
        WHERE o.CustomerID = c.CustomerID
    );

Explanation

• The subquery:

    sqlCopyEditSELECT MAX(OrderDate) FROM Orders o WHERE o.CustomerID = c.CustomerID;
  

  • Finds the most recent order date for each customer.

• The outer query:

    sqlCopyEditSELECT CustomerID, Name FROM Customers WHERE OrderDate = (subquery);
  

  • Fetches customers with their latest orders.

🔹 Example Table: Orders

👉 Output: Customer 1 (Latest order on 2024-03-20)

---

4. Find the Highest Paid Employee in Each Department

To get the highest-paid employee per department, use:

    sqlCopyEditSELECT EmployeeID, Name, Salary, DepartmentID 
    FROM Employees e1 
    WHERE Salary = (
        SELECT MAX(Salary) 
        FROM Employees e2 
        WHERE e2.DepartmentID = e1.DepartmentID
    );

How It Works

• The subquery:

    sqlCopyEditSELECT MAX(Salary) FROM Employees e2 WHERE e2.DepartmentID = e1.DepartmentID;
  

  • Fetches the maximum salary per department.

• The outer query:

    sqlCopyEditSELECT EmployeeID, Name, Salary FROM Employees WHERE Salary = (subquery);
  

  • Filters employees earning this maximum salary.

🔹 Example Table: Employees

👉 Output: Employees Jane and Mark (Highest-paid in their departments)

---

5. Identify Duplicate Employee Names

To find duplicate employee names, use:

    sqlCopyEditSELECT EmployeeID, Name 
    FROM Employees e1 
    WHERE EXISTS (
        SELECT 1 
        FROM Employees e2 
        WHERE e1.Name = e2.Name AND e1.EmployeeID <> e2.EmployeeID
    );

How It Works

• The subquery:

    sqlCopyEditSELECT 1 FROM Employees e2 WHERE e1.Name = e2.Name AND e1.EmployeeID <> e2.EmployeeID;
  

  • Checks if another employee with the same name exists.

• The outer query:

    sqlCopyEditWHERE EXISTS (subquery);
  

  • Retrieves employees with duplicate names.

🔹 Example Table: Employees

👉 Output: Employees 1 and 3 (Both named John)

---

6. Find Products with Prices Above Their Category’s Average

To fetch products priced above their category’s average, use:

    sqlCopyEditSELECT ProductID, Name 
    FROM Products p1 
    WHERE Price > (
        SELECT AVG(Price) 
        FROM Products p2 
        WHERE p2.CategoryID = p1.CategoryID
    );

Explanation

• The subquery:

    sqlCopyEditSELECT AVG(Price) FROM Products p2 WHERE p2.CategoryID = p1.CategoryID;
  

  • Finds the average price of each category.

• The outer query:

    sqlCopyEditWHERE Price > (subquery);
  

  • Fetches products costing more than their category’s average.

🔹 Example Table: Products

👉 Output: Laptop (Costlier than the average price of Category 1)

---

Final Thoughts

✅ Correlated subqueries are powerful for row-wise comparisons.
🔹 They execute once per row of the outer query.
⚡ While useful, they can be slower compared to JOINs, so use them wisely!

🔎 Looking for More SQL Tricks? 👉 Follow me on Hashnode for SQL performance tuning, real-world scenarios, and expert tips! 🚀

---

Mastering Common Table Expressions (CTEs) in SQL 🚀

1. What is a Common Table Expression (CTE)?

   A Common Table Expression (CTE) is a temporary result set that can be referenced within a SELECT, INSERT, UPDATE, or DELETE statement. It is defined using the WITH clause and helps simplify complex queries.

   🔹 CTE vs. Subquery vs. Temporary Table

   | Feature | CTE | Subquery | Temporary Table | | --- | --- | --- | --- | | Scope | Available only in the current query | Embedded in SELECT, WHERE, etc. | Exists until the session ends | | Reusability | Can be referenced multiple times | Used once inside a query | Can be queried multiple times | | Readability | Improves readability | Can be complex | Used for storing intermediate results |

   ---

   1. Basic CTE Syntax

    sqlCopyEditWITH CTE_Name (Column1, Column2, ...) AS (
        -- CTE Query
        SELECT Column1, Column2, ...
        FROM TableName
    )
    -- Use the CTE
    SELECT * FROM CTE_Name;
   

   👉 CTEs are especially useful for recursive queries, aggregations, and breaking down complex queries.

   ---

   2. Example: Find Employees Earning Above the Company’s Average Salary

   To list employees earning more than the company’s average salary, use:

    sqlCopyEditWITH AvgSalary AS (
        SELECT AVG(Salary) AS CompanyAvg
        FROM Employees
    )
    SELECT EmployeeID, Name, Salary 
    FROM Employees, AvgSalary 
    WHERE Salary > CompanyAvg;
   

   🔹 How It Works

   • The CTE (AvgSalary) calculates the company-wide average salary.

   • The main query fetches employees earning above this average.

💡 Why Use a CTE?

• This query improves readability.

• The average salary is calculated once instead of in each row evaluation.

---

3. Recursive CTE: Find Employees in a Hierarchical Reporting Structure

CTEs support recursion, making them useful for hierarchical queries (e.g., employee-manager relationships).

        sqlCopyEditWITH EmployeeHierarchy AS (
            -- Base Case: Select Top-Level Managers
            SELECT EmployeeID, Name, ManagerID
            FROM Employees
            WHERE ManagerID IS NULL
            UNION ALL
            -- Recursive Case: Find Employees Reporting to Managers
            SELECT e.EmployeeID, e.Name, e.ManagerID
            FROM Employees e
            INNER JOIN EmployeeHierarchy eh ON e.ManagerID = eh.EmployeeID
        )
        SELECT * FROM EmployeeHierarchy;

🔹 How It Works

• The base case selects top-level managers.

• The recursive case retrieves employees reporting to those managers.

• The recursion continues until all employees in the hierarchy are listed.

🔹 Example Table: Employees

👉 Output:

💡 Why Use a Recursive CTE?

• Avoids loops in SQL.

• Simplifies recursive queries that are difficult with JOINS.

---

4. Using Multiple CTEs in a Single Query

CTEs support multiple definitions, allowing step-wise data processing.

        sqlCopyEditWITH SalesPerEmployee AS (
            SELECT EmployeeID, SUM(SaleAmount) AS TotalSales
            FROM Sales
            GROUP BY EmployeeID
        ), 
        HighPerformers AS (
            SELECT EmployeeID, TotalSales 
            FROM SalesPerEmployee 
            WHERE TotalSales > 10000
        )
        SELECT * FROM HighPerformers;

🔹 How It Works

1. SalesPerEmployee CTE calculates total sales per employee.

2. HighPerformers CTE filters employees with sales above 10,000.

3. Final query retrieves high-performing employees.

💡 Why Use Multiple CTEs?

• Breaks down complex logic into smaller, readable parts.

• Improves maintainability.

---

5. Finding the Top 3 Highest Paid Employees in Each Department

Using CTEs with RANK(), we can list the top 3 highest-paid employees per department.

        sqlCopyEditWITH RankedSalaries AS (
            SELECT Name, DepartmentID, Salary,
                   RANK() OVER (PARTITION BY DepartmentID ORDER BY Salary DESC) AS Rank
            FROM Employees
        )
        SELECT Name, DepartmentID, Salary
        FROM RankedSalaries
        WHERE Rank <= 3;

🔹 How It Works

• The CTE RankedSalaries assigns ranking within each department.

• The final query filters the top 3 salaries.

💡 Why Use a CTE Here?

• Simplifies ranking logic.

• Improves performance compared to using nested subqueries.

---

6. Running Totals Using a CTE

To compute running totals for sales transactions, use:

        sqlCopyEditWITH RunningTotal AS (
            SELECT SalespersonID, SaleDate, SaleAmount,
                   SUM(SaleAmount) OVER (PARTITION BY SalespersonID ORDER BY SaleDate) AS RunningTotal
            FROM Sales
        )
        SELECT * FROM RunningTotal;

🔹 How It Works

• Uses window functions within a CTE.

• Computes cumulative sales per salesperson.

💡 Why Use a CTE Here?

• Simplifies aggregation queries.

• Enhances performance with window functions.

---

7. Deleting Data Using a CTE

To delete employees earning below $30,000, use:

        sqlCopyEditWITH LowSalaryEmployees AS (
            SELECT EmployeeID FROM Employees WHERE Salary < 30000
        )
        DELETE FROM Employees
        WHERE EmployeeID IN (SELECT EmployeeID FROM LowSalaryEmployees);

💡 Why Use a CTE?

• Prepares the dataset before deleting.

• Makes debugging easier by previewing affected records.

---

8. Updating Data Using a CTE

To increase salaries in the Sales department by 10%, use:

        sqlCopyEditWITH SalesEmployees AS (
            SELECT EmployeeID, Salary FROM Employees WHERE Department = 'Sales'
        )
        UPDATE Employees
        SET Salary = Salary * 1.1
        WHERE EmployeeID IN (SELECT EmployeeID FROM SalesEmployees);

💡 Why Use a CTE?

• Identifies employees before updating.

• Prevents unintended updates.

---

9. CTEs vs. Temporary Tables

---

Final Thoughts

✅ CTEs make SQL queries more readable, maintainable, and modular.
✅ They are great for hierarchical data, aggregations, and recursive queries.
✅ Unlike temp tables, CTEs don’t persist beyond a single query execution.

💡 When to Use CTEs? ✔️ When you need better query readability.
✔️ When performing recursive queries.
✔️ When breaking down complex logic into multiple steps.

---

🔎 Looking for More SQL Performance Tips?
👉 Follow me on Hashnode for expert SQL optimization tricks, real-world scenarios, and advanced database queries! 🚀

---

Mastering Advanced Joins in SQL: Self Joins, Cross Joins, and Full Outer Joins 🚀

1. What are Joins in SQL?

   Joins in SQL combine rows from two or more tables based on a related column. The most commonly used joins are: ✔️ INNER JOIN - Matches rows from both tables
   ✔️ LEFT JOIN - Includes all rows from the left table, and matching rows from the right
   ✔️ RIGHT JOIN - Includes all rows from the right table, and matching rows from the left
   ✔️ FULL OUTER JOIN - Returns all rows from both tables
   ✔️ SELF JOIN - A table joins itself
   ✔️ CROSS JOIN - Cartesian product of both tables

   ---

   1. Self Join: Relating a Table to Itself

   A Self Join is when a table joins with itself to compare its own rows. This is useful for hierarchical data, such as employees and managers.

   Example: Find Employees and Their Managers

    sqlCopyEditSELECT e1.EmployeeID, e1.Name AS Employee, 
           e2.Name AS Manager
    FROM Employees e1
    LEFT JOIN Employees e2 
    ON e1.ManagerID = e2.EmployeeID;
   

   How It Works

   • e1 represents employees.

   • e2 represents managers (same table, but different alias).

   • Self Join links employees with their managers using ManagerID.

🔹 Example Table: Employees

👉 Output:

💡 Use Cases for Self Joins

• Organizational hierarchy (Employee-Manager)

• Finding duplicate records

• Comparing successive rows (e.g., Price changes over time)

---

2. Cross Join: Cartesian Product of Two Tables

A Cross Join produces all possible combinations of rows between two tables. This is useful for combining categories, test scenarios, or generating data permutations.

Example: Generate All Product-Region Combinations

            sqlCopyEditSELECT p.ProductName, r.RegionName
            FROM Products p
            CROSS JOIN Regions r;

How It Works

• No ON condition is required.

• Returns all combinations of rows from Products and Regions.

🔹 Example Table: Products

🔹 Example Table: Regions

👉 Output: (All Possible Combinations)

💡 Use Cases for Cross Joins

• Generating test data (e.g., pairing users with all test scenarios)

• Creating matrix-style reports

• Calculating possible combinations of items

---

3. Full Outer Join: Include All Matching and Non-Matching Rows

A Full Outer Join returns: ✔️ Matching rows from both tables
✔️ Non-matching rows from the left table (NULLs for missing right-side values)
✔️ Non-matching rows from the right table (NULLs for missing left-side values)

Example: Retrieve All Customers and Their Orders

            sqlCopyEditSELECT c.CustomerID, c.Name, o.OrderID, o.OrderDate
            FROM Customers c
            FULL OUTER JOIN Orders o
            ON c.CustomerID = o.CustomerID;

How It Works

• Returns all customers (even if they have no orders).

• Returns all orders (even if there’s no matching customer).

• NULL values appear where there is no match.

🔹 Example Table: Customers

🔹 Example Table: Orders

👉 Output:

💡 Use Cases for Full Outer Joins

• Finding unmatched data in two tables (e.g., customers with no orders)

• Merging datasets where both sides contain important information

• Comparing differences between two data sources

---

4. Comparing Join Types with Visual Representation

---

5. Advanced Join Optimization Tips

✔️ Use indexes on foreign keys to speed up joins.
✔️ Minimize unnecessary columns in SELECT to reduce memory usage.
✔️ Prefer INNER JOINs over OUTER JOINs when possible for performance.
✔️ Use EXISTS() instead of JOIN when only checking for existence.

---

Final Thoughts

✅ Self Joins are useful for hierarchical data.
✅ Cross Joins generate all possible row combinations.
✅ Full Outer Joins ensure no data is lost from either table.

📌 When to Use Each Join? ✔️ Use INNER JOIN when both sides must have matching data.
✔️ Use LEFT JOIN when you need all data from the left table.
✔️ Use FULL OUTER JOIN when you need all data from both tables.
✔️ Use SELF JOIN for hierarchical relationships like employees & managers.
✔️ Use CROSS JOIN for all possible row combinations.

---

🔎 Want More SQL Optimization Tips?
👉 Follow me on Hashnode for real-world SQL performance tricks, database optimization tips, and hands-on SQL challenges! 🚀

---

1. If you're preparing for SQL interviews, expect questions on Joins, especially Self Joins, Cross Joins, and Full Outer Joins. Below are real-world SQL interview questions with detailed answers.

   ---

   1. What is a Self Join? Provide an Example.

   Question:

   Explain Self Joins and write a SQL query to find employees and their managers from an Employees table.

   Answer:

   A Self Join is when a table joins with itself using table aliases.

    sqlCopyEditSELECT e1.EmployeeID, e1.Name AS Employee, 
           e2.Name AS Manager
    FROM Employees e1
    LEFT JOIN Employees e2 
    ON e1.ManagerID = e2.EmployeeID;
   

   🔹 Example Table: Employees

   | EmployeeID | Name | ManagerID | | --- | --- | --- | | 1 | Alice | NULL | | 2 | Bob | 1 | | 3 | Charlie | 2 | | 4 | David | 2 |

   👉 Output:

   | EmployeeID | Employee | Manager | | --- | --- | --- | | 2 | Bob | Alice | | 3 | Charlie | Bob | | 4 | David | Bob |

   💡 Key Concept:

   • Self Joins are used for hierarchical data (e.g., employees and managers).

---

2. What is the Difference Between a Cross Join and an Inner Join?

Question:

Explain the difference between CROSS JOIN and INNER JOIN with an example.

Answer:

Example:

                sqlCopyEdit-- CROSS JOIN (All possible combinations)
                SELECT p.ProductName, r.RegionName
                FROM Products p
                CROSS JOIN Regions r;

                -- INNER JOIN (Only matching rows)
                SELECT p.ProductName, r.RegionName
                FROM Products p
                INNER JOIN Regions r ON p.RegionID = r.RegionID;

🔹 Example Table: Products

🔹 Example Table: Regions

👉 CROSS JOIN Output (All Combinations):

👉 INNER JOIN Output (Only Matches if RegionID Exists in Both Tables):

💡 Key Concept:

• Use CROSS JOIN for combinations (e.g., all product-location pairs).

• Use INNER JOIN when rows must match.

---

3. Write a Query to Find Customers Who Have Never Placed an Order

Question:

Find customers who have never placed an order using a FULL OUTER JOIN.

Answer:

                sqlCopyEditSELECT c.CustomerID, c.Name 
                FROM Customers c
                LEFT JOIN Orders o 
                ON c.CustomerID = o.CustomerID
                WHERE o.CustomerID IS NULL;

🔹 Example Table: Customers

🔹 Example Table: Orders

👉 Output (Customers with No Orders):

💡 Key Concept:

• LEFT JOIN with NULL filtering finds unmatched records (Customers without orders).

---

4. Write a Query to List All Customers and Orders, Including Customers with No Orders

Question:

Use FULL OUTER JOIN to list all customers and orders.

Answer:

                sqlCopyEditSELECT c.CustomerID, c.Name, o.OrderID, o.OrderDate
                FROM Customers c
                FULL OUTER JOIN Orders o
                ON c.CustomerID = o.CustomerID;

👉 Output:

💡 Key Concept:

• FULL OUTER JOIN ensures no data is lost from either table.

---

5. Find the Second-Highest Salary Using a Self Join

Question:

Write a SQL query to find the second-highest salary using Self Join.

Answer:

                sqlCopyEditSELECT DISTINCT e1.Salary 
                FROM Employees e1 
                WHERE 1 = (SELECT COUNT(DISTINCT e2.Salary) 
                           FROM Employees e2 
                           WHERE e2.Salary > e1.Salary);

🔹 Example Table: Employees

👉 Output: 7000 (Second-Highest Salary)

💡 Key Concept:

• Self Join helps compare values within the same table.

---

6. Find Duplicate Employee Names Using Self Join

Question:

Write a SQL query to find employees with duplicate names.

Answer:

                sqlCopyEditSELECT e1.EmployeeID, e1.Name 
                FROM Employees e1 
                WHERE EXISTS (
                    SELECT 1 FROM Employees e2 
                    WHERE e1.Name = e2.Name 
                    AND e1.EmployeeID <> e2.EmployeeID
                );

🔹 Example Table: Employees

👉 Output:

💡 Key Concept:

• Self Join helps detect duplicate values.

---

Final Thoughts

💡 Interview Takeaways: ✔️ Self Join is useful for hierarchical queries & duplicate detection.
✔️ Cross Join returns all possible row combinations.
✔️ Full Outer Join returns all matching and non-matching rows.
✔️ Use LEFT JOIN with NULL filtering to find unmatched records.
✔️ Self Join can be used to find second-highest values.

🔎 Want More SQL Interview Questions?
👉 Follow me on Hashnode for SQL coding challenges, real-world queries, and database optimization tricks! 🚀

---

1. Mastering SQL Window Functions: ROW_NUMBER, RANK, DENSE_RANK, NTILE 🚀

   Introduction to Window Functions

   SQL Window Functions are powerful tools for performing calculations across a subset of rows related to the current row, without collapsing data like GROUP BY. These functions enable ranking, aggregation, and analytical operations over a defined window of rows.

   💡 Why Use Window Functions?
   ✔️ Perform aggregations without collapsing rows
   ✔️ Rank data dynamically (e.g., Top N Customers, Employee Salaries)
   ✔️ Partition data for grouped calculations (e.g., Per Department, Per Region)
   ✔️ Easily access previous and next row values

   ---

   1. ROW_NUMBER(): Assigns Unique Row Numbers

   The ROW_NUMBER() function assigns a unique sequential number to each row within a partition, ordered by a specific column.

   💡 Use Case: Find the Latest Hired Employee in Each Department

    sqlCopyEditWITH RankedEmployees AS (
        SELECT EmployeeID, Name, DepartmentID, HireDate,
               ROW_NUMBER() OVER (PARTITION BY DepartmentID ORDER BY HireDate DESC) AS RowNum
        FROM Employees
    )
    SELECT EmployeeID, Name, DepartmentID, HireDate
    FROM RankedEmployees
    WHERE RowNum = 1;
   

   How It Works

   • PARTITION BY DepartmentID → Resets numbering per department

   • ORDER BY HireDate DESC → Assigns ROW_NUMBER based on latest hire date

   • Filters only rows with RowNum = 1, i.e., the latest hired employee per department

🔹 Example Table: Employees

👉 Output:

---

2. RANK(): Handles Ties by Assigning the Same Rank

The RANK() function assigns a ranking to rows based on sorting order, but skips ranks for duplicates.

💡 Use Case: Find the Top 3 Highest Salaries Per Department

                    sqlCopyEditWITH RankedSalaries AS (
                        SELECT EmployeeID, Name, DepartmentID, Salary,
                               RANK() OVER (PARTITION BY DepartmentID ORDER BY Salary DESC) AS SalaryRank
                        FROM Employees
                    )
                    SELECT EmployeeID, Name, DepartmentID, Salary, SalaryRank
                    FROM RankedSalaries
                    WHERE SalaryRank <= 3;

How It Works

• PARTITION BY DepartmentID → Resets ranking for each department

• ORDER BY Salary DESC → Assigns ranks based on highest salary

• WHERE SalaryRank <= 3 → Filters top 3 salaries per department

🔹 Example Table: Employees

👉 Output:

💡 Key Observation:

• Ranks 2 and 2 for the same salary

• Skipping 3 (next rank is 4, not 3)

---

3. DENSE_RANK(): Assigns Consecutive Ranks Without Skipping

The DENSE_RANK() function works like RANK() but does not skip ranks for ties.

💡 Use Case: Find the Top 3 Best-Selling Products Per Category

                    sqlCopyEditWITH RankedProducts AS (
                        SELECT ProductID, ProductName, CategoryID, TotalSales,
                               DENSE_RANK() OVER (PARTITION BY CategoryID ORDER BY TotalSales DESC) AS SalesRank
                        FROM SalesData
                    )
                    SELECT ProductID, ProductName, CategoryID, TotalSales, SalesRank
                    FROM RankedProducts
                    WHERE SalesRank <= 3;

🔹 Example Table: SalesData

👉 Output:

💡 Key Observation:

• Same rank (2) for same sales value

• Next rank is 3 (not skipping like RANK() does)

---

4. NTILE(): Dividing Data into Equal Buckets

The NTILE(n) function splits rows into n equal groups.

💡 Use Case: Divide Employees into 4 Performance Quartiles Based on Sales

                    sqlCopyEditSELECT EmployeeID, Name, Sales, 
                           NTILE(4) OVER (ORDER BY Sales DESC) AS Quartile
                    FROM SalesData;

🔹 Example Table: SalesData

👉 Output (Quartiles Assigned)

💡 Key Concept:

• Ranks employees into 4 quartiles

• Useful for percentile-based ranking

---

Comparing Window Functions: Key Differences

---

Final Thoughts

✅ Window Functions are powerful for ranking, partitioning, and analytical queries.
✅ Use ROW_NUMBER() for unique sequential ranking.
✅ Use RANK() when skipping ranks for ties.
✅ Use DENSE_RANK() when consecutive ranking is needed.
✅ Use NTILE(n) for splitting data into groups.

🔎 Want More SQL Performance Tips?
👉 Follow me on Hashnode for real-world SQL challenges, optimizations, and advanced techniques! 🚀

---

Mastering SQL Aggregate Functions with PARTITION BY 🚀

1. Introduction to Aggregate Functions with PARTITION BY

   SQL Aggregate Functions (SUM(), AVG(), COUNT(), MAX(), MIN()) summarize data. By default, they collapse multiple rows into one. However, when used with PARTITION BY, they preserve row-level granularity while grouping within partitions.

   💡 Why Use PARTITION BY?
   ✔️ Aggregate within groups without losing individual rows
   ✔️ Calculate running totals, ranks, and averages per group
   ✔️ Perform advanced analytics without GROUP BY collapsing data

   ---

   1. Understanding PARTITION BY with Aggregate Functions

   A standard aggregate function collapses data:

    sqlCopyEditSELECT DepartmentID, AVG(Salary) AS AvgSalary
    FROM Employees
    GROUP BY DepartmentID;
   

   🔹 Output (Collapses into One Row Per Department)

   | DepartmentID | AvgSalary | | --- | --- | | 101 | 7500 | | 102 | 6500 |

   ---

   How PARTITION BY Works

   Using PARTITION BY, each row is preserved, and the aggregate function is calculated for each partition:

    sqlCopyEditSELECT EmployeeID, Name, DepartmentID, Salary,
           AVG(Salary) OVER (PARTITION BY DepartmentID) AS AvgSalary
    FROM Employees;
   

   🔹 Example Table: Employees

   | EmployeeID | Name | DepartmentID | Salary | | --- | --- | --- | --- | | 1 | John | 101 | 8000 | | 2 | Jane | 101 | 7000 | | 3 | Mark | 102 | 6000 | | 4 | Sam | 102 | 7000 |

   👉 Output (No Data Loss!)

   | EmployeeID | Name | DepartmentID | Salary | AvgSalary | | --- | --- | --- | --- | --- | | 1 | John | 101 | 8000 | 7500 | | 2 | Jane | 101 | 7000 | 7500 | | 3 | Mark | 102 | 6000 | 6500 | | 4 | Sam | 102 | 7000 | 6500 |

   💡 Key Concept:

   • AVG(Salary) OVER (PARTITION BY DepartmentID) → Computes average salary per department.

   • Unlike GROUP BY, PARTITION BY keeps all rows.

---

2. Using SUM() with PARTITION BY

💡 Use Case: Calculate Total Sales Per Region

                        sqlCopyEditSELECT OrderID, RegionID, Sales,
                               SUM(Sales) OVER (PARTITION BY RegionID) AS TotalSalesPerRegion
                        FROM Orders;

🔹 Example Table: Orders

👉 Output (Total Sales Per Region for Each Row)

💡 Key Concept:

• Calculates total sales per RegionID without collapsing rows.

---

3. Using COUNT() with PARTITION BY

💡 Use Case: Count Number of Employees Per Department

                        sqlCopyEditSELECT EmployeeID, Name, DepartmentID, 
                               COUNT(*) OVER (PARTITION BY DepartmentID) AS EmployeeCount
                        FROM Employees;

🔹 Output:

💡 Key Concept:

• COUNT(*) OVER (PARTITION BY DepartmentID) counts employees per department without collapsing data.

---

4. Using MAX() and MIN() with PARTITION BY

💡 Use Case: Find Highest & Lowest Salary Per Department

                        sqlCopyEditSELECT EmployeeID, Name, DepartmentID, Salary,
                               MAX(Salary) OVER (PARTITION BY DepartmentID) AS MaxSalary,
                               MIN(Salary) OVER (PARTITION BY DepartmentID) AS MinSalary
                        FROM Employees;

🔹 Output:

💡 Key Concept:

• Determines max & min salaries per department without collapsing rows.

---

5. Using AVG() for Moving Averages

💡 Use Case: Find 3-Month Moving Average Sales Per Region

                        sqlCopyEditSELECT OrderID, RegionID, Sales, OrderDate,
                               AVG(Sales) OVER (PARTITION BY RegionID ORDER BY OrderDate ROWS BETWEEN 2 PRECEDING AND CURRENT ROW) AS MovingAvg
                        FROM Orders;

🔹 Output:

💡 Key Concept:

• Calculates a 3-month rolling average sales per region.

---

6. Combining Multiple Aggregates in One Query

                        sqlCopyEditSELECT EmployeeID, Name, DepartmentID, Salary,
                               SUM(Salary) OVER (PARTITION BY DepartmentID) AS TotalDeptSalary,
                               AVG(Salary) OVER (PARTITION BY DepartmentID) AS AvgDeptSalary,
                               COUNT(*) OVER (PARTITION BY DepartmentID) AS EmployeeCount
                        FROM Employees;

🔹 Output:

💡 Key Concept:

• Combining multiple partitioned aggregates in one query.

---

Final Thoughts

✅ PARTITION BY preserves rows while grouping data dynamically.
✅ Use SUM(), AVG(), COUNT(), MAX(), MIN() for analytics per partition.
✅ Combine multiple aggregates in a single query for detailed insights.

🔎 Want More SQL Performance Tips?
👉 Follow me on Hashnode for real-world SQL challenges, optimizations, and advanced techniques! 🚀

---

Mastering GROUPING SETS, CUBE, and ROLLUP in SQL 🚀

Introduction to Advanced Aggregation

SQL provides advanced aggregation techniques to analyze data at multiple levels using:

1. GROUPING SETS - Define multiple grouping combinations in a single query.

2. ROLLUP - Aggregate data hierarchically (drill-down analysis).

3. CUBE - Aggregate data for all possible combinations.

💡 Why Use These Techniques? ✔️ Perform multiple GROUP BY aggregations in one query
✔️ Improve performance vs running multiple GROUP BY queries
✔️ Easier hierarchical & multi-dimensional analysis

---

1. Understanding GROUPING SETS

What is GROUPING SETS?

GROUPING SETS allows defining multiple GROUP BY aggregations in one query, without requiring multiple queries.

💡 Use Case: Find Total Sales Per Region & Product

                        sqlCopyEditSELECT RegionID, ProductID, SUM(Sales) AS TotalSales
                        FROM Sales
                        GROUP BY GROUPING SETS (
                            (RegionID),        -- Total sales per region
                            (ProductID),       -- Total sales per product
                            (RegionID, ProductID) -- Total sales per region-product combination
                        );

🔹 Example Table: Sales

👉 Output (Grouped Aggregation)

💡 Key Concept:

• Returns aggregated results for each grouping set.

• NULL represents totals for that level.

---

2. Understanding ROLLUP

What is ROLLUP?

ROLLUP creates hierarchical aggregations, where each level builds on the previous one.

💡 Use Case: Find Total Sales Per Region, Product, and Grand Total

                        sqlCopyEditSELECT RegionID, ProductID, SUM(Sales) AS TotalSales
                        FROM Sales
                        GROUP BY ROLLUP (RegionID, ProductID);

👉 Output (Hierarchical Aggregation)

💡 Key Concept:

• ROLLUP(RegionID, ProductID) creates a hierarchy:
  ✔️ Region + Product
  ✔️ Region (Subtotal)
  ✔️ Grand Total

---

3. Understanding CUBE

What is CUBE?

CUBE computes all possible groupings, including cross-combinations.

💡 Use Case: Find Sales for All Combinations of Region & Product

                        sqlCopyEditSELECT RegionID, ProductID, SUM(Sales) AS TotalSales
                        FROM Sales
                        GROUP BY CUBE (RegionID, ProductID);

👉 Output (All Combinations)

💡 Key Concept:

• CUBE(RegionID, ProductID) creates all possible combinations, including:
  ✔️ Region + Product
  ✔️ Region Total
  ✔️ Product Total
  ✔️ Grand Total

---

4. Comparing GROUPING SETS vs ROLLUP vs CUBE

---

5. Handling NULL Values with GROUPING()

💡 GROUPING() helps differentiate NULL values for subtotals.

                        sqlCopyEditSELECT 
                            CASE 
                                WHEN GROUPING(RegionID) = 1 THEN 'All Regions'
                                ELSE CAST(RegionID AS VARCHAR)
                            END AS Region,
                            CASE 
                                WHEN GROUPING(ProductID) = 1 THEN 'All Products'
                                ELSE CAST(ProductID AS VARCHAR)
                            END AS Product,
                            SUM(Sales) AS TotalSales
                        FROM Sales
                        GROUP BY CUBE (RegionID, ProductID);

👉 Output (With Labels)

---

Final Thoughts

✅ GROUPING SETS → Define custom groupings.
✅ ROLLUP → Hierarchical subtotal calculations.
✅ CUBE → All possible aggregations including cross-combinations.
✅ GROUPING() → Helps label NULL values for better readability.

🔎 Want More SQL Optimization Tips?
👉 Follow me on Hashnode for real-world SQL challenges, optimizations, and advanced analytics! 🚀

Variables and Data Types in T-SQL (Transact-SQL) 🚀

Introduction to Variables in T-SQL

In T-SQL (Transact-SQL), variables are temporary storage objects that hold values during query execution. They are useful for storing values, passing data, and enhancing query performance.

1. 💡 Why Use Variables in T-SQL?
   ✔️ Store intermediate values for calculations
   ✔️ Pass values dynamically into queries
   ✔️ Optimize performance by reducing redundant calculations
   ✔️ Improve readability and maintainability of code

   ---

   1. Declaring and Assigning Variables

   In T-SQL, variables are declared using the DECLARE statement and assigned values using SET or SELECT.

   Basic Syntax:

    sqlCopyEditDECLARE @VariableName DataType;
    SET @VariableName = Value;
   

   ---

   Example: Declaring and Assigning a Variable

    sqlCopyEditDECLARE @EmployeeName NVARCHAR(100);
    SET @EmployeeName = 'John Doe';
   
    PRINT @EmployeeName; -- Output: John Doe
   

   💡 Key Points:

   • @EmployeeName is a local variable.

   • NVARCHAR(100) specifies a string data type.

   • PRINT displays variable value.

---

2. Assigning Values Using SET vs SELECT

T-SQL allows assigning values using:

1. SET (Best for single-value assignments)

2. SELECT (Faster for multiple-value assignments)

Using SET (Preferred for Single Value Assignments)

                        sqlCopyEditDECLARE @Salary INT;
                        SET @Salary = 50000;

                        PRINT @Salary; -- Output: 50000

Using SELECT (Assign from Query Result)

                        sqlCopyEditDECLARE @MaxSalary INT;
                        SELECT @MaxSalary = MAX(Salary) FROM Employees;

                        PRINT @MaxSalary; -- Output: (Highest Salary)

💡 Difference Between SET and SELECT

---

3. Using Variables in Queries

Example: Dynamic Query with Variables

                        sqlCopyEditDECLARE @DepartmentID INT;
                        SET @DepartmentID = 101;

                        SELECT EmployeeID, Name, Salary 
                        FROM Employees 
                        WHERE DepartmentID = @DepartmentID;

💡 Key Benefit:

• Dynamic queries make queries reusable and flexible.

---

4. Table Variables: Storing Temporary Data

A table variable stores temporary table-like data.

Example: Using a Table Variable

                        sqlCopyEditDECLARE @TempTable TABLE (
                            EmployeeID INT,
                            Name NVARCHAR(100),
                            Salary INT
                        );

                        INSERT INTO @TempTable VALUES (1, 'Alice', 6000), (2, 'Bob', 7000);

                        SELECT * FROM @TempTable;

💡 Key Differences Between Table Variables and Temp Tables

---

5. Understanding Data Types in T-SQL

T-SQL supports various data types for storing different types of data.

Commonly Used Data Types

1. Numeric Data Types

Example: Declaring Numeric Variables

                        sqlCopyEditDECLARE @OrderAmount DECIMAL(10,2);
                        SET @OrderAmount = 1250.75;

                        PRINT @OrderAmount; -- Output: 1250.75

---

2. String (Character) Data Types

Example: Declaring String Variables

                        sqlCopyEditDECLARE @CustomerName NVARCHAR(50);
                        SET @CustomerName = 'Alice';

                        PRINT @CustomerName; -- Output: Alice

---

3. Date and Time Data Types

Example: Storing Dates

                        sqlCopyEditDECLARE @CurrentDate DATE;
                        SET @CurrentDate = GETDATE();

                        PRINT @CurrentDate; -- Output: 2024-02-12 (or today's date)

---

4. Boolean Data Type

T-SQL does not have a direct BOOLEAN type. Instead, we use BIT.

Example: Using BIT for Boolean Logic

                        sqlCopyEditDECLARE @IsActive BIT;
                        SET @IsActive = 1;

                        PRINT @IsActive; -- Output: 1

---

6. Using Variables in Stored Procedures

Variables are commonly used in stored procedures.

Example: Stored Procedure Using Variables

                        sqlCopyEditCREATE PROCEDURE GetEmployeesByDept
                            @DeptID INT
                        AS
                        BEGIN
                            SELECT EmployeeID, Name, Salary
                            FROM Employees
                            WHERE DepartmentID = @DeptID;
                        END;

💡 Calling the Procedure

                        sqlCopyEditEXEC GetEmployeesByDept @DeptID = 101;

---

7. Dynamic SQL Using Variables

Dynamic SQL generates queries dynamically at runtime.

Example: Dynamic SQL with Variables

                        sqlCopyEditDECLARE @SQLQuery NVARCHAR(MAX);
                        DECLARE @TableName NVARCHAR(100) = 'Employees';

                        SET @SQLQuery = 'SELECT * FROM ' + @TableName;
                        EXEC sp_executesql @SQLQuery;

💡 Why Use Dynamic SQL?
✔️ Flexibility – Query can be modified dynamically
✔️ Parameterization – Safer from SQL injection

---

8. Global Variables in T-SQL

SQL Server provides system-level global variables.

Common System Variables

Example: Using @@IDENTITY

                        sqlCopyEditINSERT INTO Orders (CustomerID, OrderAmount) VALUES (1, 500);
                        SELECT @@IDENTITY AS LastInsertedID;

---

Final Thoughts

✅ Variables in T-SQL store temporary values for calculations.
✅ Use DECLARE + SET or SELECT to assign values.
✅ Table Variables (@TableVar) are lightweight for temporary storage.
✅ Data types define how data is stored and processed.
✅ Dynamic SQL makes queries flexible and dynamic.

🔎 Want More SQL Performance Tips?
👉 Follow me on Hashnode for real-world SQL challenges, optimizations, and best practices! 🚀

---

Mastering Dynamic SQL and sp_executesql in T-SQL 🚀

Introduction to Dynamic SQL

Dynamic SQL allows query construction at runtime, making it useful for dynamic filtering, conditional execution, and handling variable table names or columns.

💡 Why Use Dynamic SQL?
✔️ Flexibility – Modify SQL queries dynamically
✔️ Parameterized Queries – Prevent SQL injection
✔️ Handle Dynamic Filtering – Build flexible search queries
✔️ Query Optimization – Reduce unnecessary query execution

---

1. What is Dynamic SQL?

Dynamic SQL is a SQL statement constructed as a string and executed using:

1. EXEC() (Basic Execution)

2. sp_executesql (Secure & Optimized)

Basic Syntax:

                        sqlCopyEditDECLARE @SQLQuery NVARCHAR(MAX);
                        SET @SQLQuery = 'SELECT * FROM Employees WHERE DepartmentID = 101';
                        EXEC(@SQLQuery);

💡 Limitations of EXEC()

• No support for parameterized queries (vulnerable to SQL injection).

• No query plan reuse (performance issues).

---

2. Using sp_executesql: The Secure and Efficient Way

sp_executesql executes dynamic SQL with parameters, improving security and performance.

Basic Syntax:

                        sqlCopyEditDECLARE @SQLQuery NVARCHAR(MAX);
                        DECLARE @DeptID INT = 101;

                        SET @SQLQuery = 'SELECT * FROM Employees WHERE DepartmentID = @DeptID';
                        EXEC sp_executesql @SQLQuery, N'@DeptID INT', @DeptID;

💡 Key Benefits of sp_executesql Over EXEC()

---

3. Using Dynamic SQL for Flexible Search Queries

💡 Use Case: Search Employees Dynamically Based on Optional Filters

                        sqlCopyEditDECLARE @SQLQuery NVARCHAR(MAX);
                        DECLARE @EmployeeName NVARCHAR(100) = 'John';
                        DECLARE @Salary INT = NULL; -- Optional parameter

                        SET @SQLQuery = 'SELECT * FROM Employees WHERE 1=1';

                        IF @EmployeeName IS NOT NULL
                            SET @SQLQuery = @SQLQuery + ' AND Name = @EmployeeName';

                        IF @Salary IS NOT NULL
                            SET @SQLQuery = @SQLQuery + ' AND Salary > @Salary';

                        EXEC sp_executesql @SQLQuery, N'@EmployeeName NVARCHAR(100), @Salary INT', @EmployeeName, @Salary;

🔹 How It Works

• Flexible Query: The query dynamically appends conditions based on user input.

• Prevents SQL Injection: Uses parameterized queries instead of string concatenation.

---

4. Using Dynamic SQL with Table Names

Sometimes, you may need to pass table names dynamically.

💡 Use Case: Querying Different Tables Dynamically

                        sqlCopyEditDECLARE @SQLQuery NVARCHAR(MAX);
                        DECLARE @TableName NVARCHAR(100) = 'Employees';

                        SET @SQLQuery = 'SELECT * FROM ' + QUOTENAME(@TableName);
                        EXEC sp_executesql @SQLQuery;

🔹 Why Use QUOTENAME()?

• Prevents SQL injection by enclosing table names in safe brackets ([ ]).

---

5. Using Dynamic SQL with OUTPUT Parameters

Dynamic SQL supports returning values using OUTPUT parameters.

💡 Use Case: Find Employee Count in a Department

                        sqlCopyEditDECLARE @SQLQuery NVARCHAR(MAX);
                        DECLARE @DeptID INT = 101;
                        DECLARE @EmployeeCount INT;

                        SET @SQLQuery = 'SELECT @Count = COUNT(*) FROM Employees WHERE DepartmentID = @DeptID';

                        EXEC sp_executesql @SQLQuery, 
                            N'@DeptID INT, @Count INT OUTPUT', 
                            @DeptID, @EmployeeCount OUTPUT;

                        PRINT @EmployeeCount; -- Outputs the employee count

🔹 How It Works

• Uses OUTPUT parameter to store query results.

• Efficiently returns values from dynamic SQL execution.

---

6. Handling SQL Injection in Dynamic SQL

Dynamic SQL can be vulnerable to SQL Injection if not parameterized.

🚨 Unsafe Example (Vulnerable to SQL Injection)

                        sqlCopyEditDECLARE @SQLQuery NVARCHAR(MAX);
                        DECLARE @DeptID NVARCHAR(50) = '101; DROP TABLE Employees --';

                        SET @SQLQuery = 'SELECT * FROM Employees WHERE DepartmentID = ' + @DeptID;
                        EXEC(@SQLQuery); -- 🚨 Dangerous!

💀 Risk: This allows malicious input (DROP TABLE Employees), leading to data loss.

---

✅ Secure Example (Using sp_executesql)

                        sqlCopyEditDECLARE @SQLQuery NVARCHAR(MAX);
                        DECLARE @DeptID INT = 101;

                        SET @SQLQuery = 'SELECT * FROM Employees WHERE DepartmentID = @DeptID';
                        EXEC sp_executesql @SQLQuery, N'@DeptID INT', @DeptID;

🔒 Why Is This Secure?

• No direct concatenation of user input.

• Uses parameterized query execution.

---

7. Using Dynamic SQL in Stored Procedures

💡 Use Case: Creating a Search Stored Procedure

                        sqlCopyEditCREATE PROCEDURE GetEmployeesByDepartment
                            @DeptID INT
                        AS
                        BEGIN
                            DECLARE @SQLQuery NVARCHAR(MAX);
                            SET @SQLQuery = 'SELECT * FROM Employees WHERE DepartmentID = @DeptID';

                            EXEC sp_executesql @SQLQuery, N'@DeptID INT', @DeptID;
                        END;

💡 Calling the Procedure

                        sqlCopyEditEXEC GetEmployeesByDepartment @DeptID = 101;

✔️ Reusable, secure, and optimized.

---

8. Using sp_executesql with Multiple Parameters

💡 Use Case: Fetch Employees by Department and Salary Range

                        sqlCopyEditDECLARE @SQLQuery NVARCHAR(MAX);
                        DECLARE @DeptID INT = 101;
                        DECLARE @MinSalary INT = 5000;
                        DECLARE @MaxSalary INT = 10000;

                        SET @SQLQuery = 'SELECT * FROM Employees WHERE DepartmentID = @DeptID AND Salary BETWEEN @MinSalary AND @MaxSalary';

                        EXEC sp_executesql @SQLQuery, 
                            N'@DeptID INT, @MinSalary INT, @MaxSalary INT', 
                            @DeptID, @MinSalary, @MaxSalary;

🔹 How It Works
✔️ Filters dynamically based on department and salary range.
✔️ Uses multiple parameters securely.

---

9. Performance Optimization with Cached Execution Plans

Dynamic SQL with sp_executesql caches execution plans, improving performance.

💡 Comparing Performance: EXEC() vs sp_executesql

                        sqlCopyEdit-- Using EXEC (Does NOT cache execution plan)
                        DECLARE @SQLQuery NVARCHAR(MAX);
                        SET @SQLQuery = 'SELECT * FROM Employees WHERE DepartmentID = 101';
                        EXEC(@SQLQuery);

                        -- Using sp_executesql (Caches execution plan)
                        DECLARE @SQLQuery NVARCHAR(MAX);
                        DECLARE @DeptID INT = 101;
                        SET @SQLQuery = 'SELECT * FROM Employees WHERE DepartmentID = @DeptID';
                        EXEC sp_executesql @SQLQuery, N'@DeptID INT', @DeptID;

💡 Why Does sp_executesql Perform Better?
✔️ Caches query execution plan for better performance
✔️ Reuses the plan for different input values

---

10. Best Practices for Using Dynamic SQL

✅ Always use sp_executesql for security & performance.
✅ Use QUOTENAME() for table and column names to prevent SQL injection.
✅ Use OUTPUT parameters if results need to be retrieved.
✅ Avoid excessive dynamic SQL where static queries suffice.
✅ Cache execution plans by using parameterized queries.

---

Final Thoughts

✅ Dynamic SQL enables flexible, dynamic queries.
✅ sp_executesql is safer and more performant than EXEC().
✅ Always parameterize input to prevent SQL injection.

🔎 Want More SQL Performance Tips?
👉 Follow me on Hashnode for real-world SQL challenges, optimizations, and best practices! 🚀

1. Understanding Transactions in SQL Server

A transaction is a sequence of operations executed as a single unit.

🔹 Types of Transactions
✔️ Implicit Transactions - SQL Server automatically starts and commits/rolls back transactions.
✔️ Explicit Transactions - Manually controlled with BEGIN TRANSACTION, COMMIT, and ROLLBACK.
✔️ Autocommit Transactions - Each SQL statement runs independently (default mode).

Basic Transaction Syntax

                    sqlCopyEditBEGIN TRANSACTION;
                    -- SQL Operations
                    COMMIT TRANSACTION; -- Saves Changes
                    ROLLBACK TRANSACTION; -- Reverts Changes

---

2. Implementing Explicit Transactions

💡 Use Case: Transferring Funds Between Accounts

                    sqlCopyEditBEGIN TRANSACTION;

                    UPDATE Accounts SET Balance = Balance - 100 WHERE AccountID = 1;
                    UPDATE Accounts SET Balance = Balance + 100 WHERE AccountID = 2;

                    COMMIT TRANSACTION;

✅ Ensures both updates succeed or both fail together.

---

3. Rolling Back Transactions on Errors

💡 Use Case: Prevent Partial Updates

                    sqlCopyEditBEGIN TRANSACTION;

                    UPDATE Employees SET Salary = Salary + 500 WHERE EmployeeID = 1;

                    -- Simulated Error
                    IF @@ERROR <> 0 
                    BEGIN
                        PRINT 'Error occurred! Rolling back transaction.';
                        ROLLBACK TRANSACTION;
                        RETURN;
                    END

                    COMMIT TRANSACTION;

💡 How It Works?
✔️ @@ERROR checks for SQL errors after an operation.
✔️ If an error occurs, ROLLBACK undoes the update.

---

4. Using TRY...CATCH for Error Handling

💡 Use Case: Handling Errors in Transactions

                    sqlCopyEditBEGIN TRANSACTION;

                    BEGIN TRY
                        -- SQL Statements
                        UPDATE Employees SET Salary = Salary + 500 WHERE EmployeeID = 1;

                        -- Simulate Error (Division by Zero)
                        SELECT 1/0 AS ErrorTest;

                        COMMIT TRANSACTION; -- If no error, commit changes

                    END TRY
                    BEGIN CATCH
                        PRINT 'Error occurred! Rolling back transaction.';
                        ROLLBACK TRANSACTION;
                    END CATCH;

💡 Key Features:
✔️ TRY...CATCH blocks handle runtime errors.
✔️ If an error occurs, CATCH executes and rolls back the transaction.

---

5. Using ERROR_MESSAGE() and ERROR_NUMBER()

Inside a CATCH block, we can capture detailed error information.

💡 Use Case: Log Errors with Details

                    sqlCopyEditBEGIN TRANSACTION;

                    BEGIN TRY
                        -- Simulate Error
                        SELECT 1/0 AS ErrorTest;

                        COMMIT TRANSACTION;
                    END TRY
                    BEGIN CATCH
                        PRINT 'Error Number: ' + CAST(ERROR_NUMBER() AS NVARCHAR);
                        PRINT 'Error Message: ' + ERROR_MESSAGE();

                        ROLLBACK TRANSACTION;
                    END CATCH;

🔹 Example Output

                    vbnetCopyEditError Number: 8134
                    Error Message: Divide by zero error encountered.

💡 Key Functions

---

6. Nested Transactions and @@TRANCOUNT

💡 Use Case: Handling Nested Transactions

                    sqlCopyEditBEGIN TRANSACTION;

                    UPDATE Employees SET Salary = Salary + 500 WHERE EmployeeID = 1;

                    BEGIN TRANSACTION; -- Nested Transaction
                    UPDATE Employees SET Salary = Salary + 500 WHERE EmployeeID = 2;

                    PRINT 'Transaction Count: ' + CAST(@@TRANCOUNT AS NVARCHAR);

                    COMMIT TRANSACTION; -- Commits Inner Transaction
                    COMMIT TRANSACTION; -- Commits Outer Transaction

💡 Why @@TRANCOUNT?
✔️ Keeps track of active transactions.
✔️ Helps in nested transactions handling.

---

7. Using SAVE TRANSACTION for Partial Rollbacks

💡 Use Case: Save Points for Partial Rollback

                    sqlCopyEditBEGIN TRANSACTION;

                    UPDATE Employees SET Salary = Salary + 500 WHERE EmployeeID = 1;
                    SAVE TRANSACTION SavePoint1; -- Save state

                    -- Simulate Error
                    UPDATE Employees SET Salary = Salary + 500 WHERE EmployeeID = NULL;

                    IF @@ERROR <> 0 
                    BEGIN
                        PRINT 'Error occurred! Rolling back to SavePoint1.';
                        ROLLBACK TRANSACTION SavePoint1; -- Partial Rollback
                    END

                    COMMIT TRANSACTION;

💡 Key Benefits:
✔️ ROLLBACK to a SavePoint instead of full rollback.
✔️ Useful for complex transactions.

---

8. Using XACT_ABORT for Automatic Rollback

🔹 XACT_ABORT automatically rolls back transactions on failure.

💡 Use Case: Ensuring Full Rollback on Error

                    sqlCopyEditSET XACT_ABORT ON;

                    BEGIN TRANSACTION;

                    UPDATE Employees SET Salary = Salary + 500 WHERE EmployeeID = 1;

                    -- Simulated Error (Divide by Zero)
                    SELECT 1/0 AS ErrorTest;

                    COMMIT TRANSACTION; -- Won't execute due to error

💡 Why Use XACT_ABORT?
✔️ Automatically rolls back transactions on errors.
✔️ Avoids partially committed transactions.

---

9. Comparing Transaction Control Commands

---

10. Best Practices for Error Handling & Transactions

✅ Always Use TRY...CATCH for Error Handling.
✅ Use sp_executesql for Dynamic SQL instead of EXEC().
✅ Enable XACT_ABORT for Automatic Rollback.
✅ Check @@TRANCOUNT Before Rolling Back Nested Transactions.
✅ Use SAVE TRANSACTION for Partial Rollbacks.

---

Final Thoughts

✅ Transactions ensure data integrity with rollback and commit.
✅ TRY...CATCH prevents SQL errors from breaking execution.
✅ Use ERROR_MESSAGE() for debugging errors efficiently.
✅ Use XACT_ABORT ON to prevent accidental partial commits.

🔎 Want More SQL Performance Tips?
👉 Follow me on Hashnode for real-world SQL challenges, optimizations, and best practices! 🚀

Introduction

In modern SQL Server development, functions and stored procedures play a crucial role in optimizing performance, enhancing reusability, and maintaining clean and efficient code. Whether you're working on complex calculations, data transformation, or procedural logic, mastering advanced functions and stored procedures is essential for scalable database design.

🔹 Why Are Functions & Stored Procedures Important?
✔️ Encapsulate Business Logic – Keep code organized and reusable.
✔️ Improve Performance – Reduce redundant calculations and enhance efficiency.
✔️ Enhance Security – Restrict direct table access while allowing controlled execution.
✔️ Optimize Query Execution – Precompiled and parameterized execution plans improve speed.

---

What You’ll Learn in This Guide

In this comprehensive guide, we will explore advanced SQL Server functions and stored procedures, including:

✅ Scalar & Table-Valued Functions – Efficient ways to process and return data.
✅ Inline vs. Multi-Statement Functions – Performance implications and best practices.
✅ Stored Procedures with Input & Output Parameters – Dynamic and flexible execution.
✅ Handling Transactions in Stored Procedures – Ensuring consistency and rollback safety.
✅ Error Handling (TRY...CATCH) in Procedures – Secure error management.
✅ Using sp_executesql for Dynamic Queries – Optimizing execution with parameterized SQL

1. Advanced Functions in T-SQL

1. Functions in SQL Server allow encapsulation of logic into reusable blocks of code. There are two main types:

   1. Scalar Functions → Returns a single value

   2. Table-Valued Functions (TVFs) → Returns a table

      • Inline Table-Valued Functions

      • Multi-Statement Table-Valued Functions

---

1.1 Scalar Functions

🔹 A Scalar Function returns a single value (e.g., GETDATE(), LEN()).

💡 Use Case: Creating a Function to Format Full Name

                        sqlCopyEditCREATE FUNCTION dbo.GetFullName(@FirstName NVARCHAR(50), @LastName NVARCHAR(50))
                        RETURNS NVARCHAR(100)
                        AS
                        BEGIN
                            RETURN @FirstName + ' ' + @LastName;
                        END;

Calling the Function

                        sqlCopyEditSELECT dbo.GetFullName('John', 'Doe') AS FullName;

👉 Output: John Doe

💡 Key Features:
✔️ Takes input parameters
✔️ Returns a single value
✔️ Cannot modify database state (INSERT, UPDATE, DELETE not allowed)

---

1.2 Table-Valued Functions (TVFs)

TVFs return a table instead of a scalar value.

Inline Table-Valued Functions

🔹 Returns a table without using BEGIN...END.

💡 Use Case: Get Employees in a Specific Department

                        sqlCopyEditCREATE FUNCTION dbo.GetEmployeesByDept(@DeptID INT)
                        RETURNS TABLE
                        AS
                        RETURN 
                        (
                            SELECT EmployeeID, Name, Salary 
                            FROM Employees 
                            WHERE DepartmentID = @DeptID
                        );

Calling the Function

                        sqlCopyEditSELECT * FROM dbo.GetEmployeesByDept(101);

💡 Why Use Inline TVFs?
✔️ Efficient execution (like a view)
✔️ Can be used in JOINs and SELECT statements

---

Multi-Statement Table-Valued Functions

🔹 Uses BEGIN...END and supports multiple statements.

💡 Use Case: Return Employees with Salary Category

                        sqlCopyEditCREATE FUNCTION dbo.GetEmployeesWithSalaryCategory()
                        RETURNS @EmployeeTable TABLE
                        (
                            EmployeeID INT, 
                            Name NVARCHAR(100),
                            Salary INT,
                            SalaryCategory NVARCHAR(20)
                        )
                        AS
                        BEGIN
                            INSERT INTO @EmployeeTable
                            SELECT EmployeeID, Name, Salary,
                                   CASE 
                                       WHEN Salary > 7000 THEN 'High' 
                                       WHEN Salary BETWEEN 4000 AND 7000 THEN 'Medium' 
                                       ELSE 'Low' 
                                   END
                            FROM Employees;

                            RETURN;
                        END;

Calling the Function

                        sqlCopyEditSELECT * FROM dbo.GetEmployeesWithSalaryCategory();

💡 When to Use Multi-Statement TVFs?
✔️ When complex logic needs multiple statements
✔️ When temporary results need modification before returning

---

1.3 Inline vs. Multi-Statement TVFs

---

2. Stored Procedures in SQL Server

Stored Procedures are precompiled SQL blocks used to execute complex logic efficiently.

2.1 Basic Stored Procedure

💡 Use Case: Get Employees by Department

                        sqlCopyEditCREATE PROCEDURE GetEmployeesByDept
                            @DeptID INT
                        AS
                        BEGIN
                            SELECT EmployeeID, Name, Salary
                            FROM Employees
                            WHERE DepartmentID = @DeptID;
                        END;

Calling the Stored Procedure

                        sqlCopyEditEXEC GetEmployeesByDept @DeptID = 101;

💡 Why Use Stored Procedures?
✔️ Precompiled for Faster Execution
✔️ Supports Input & Output Parameters
✔️ Encapsulates Business Logic

---

2.2 Stored Procedure with Multiple Parameters

💡 Use Case: Get Employees by Department and Salary Range

                        sqlCopyEditCREATE PROCEDURE GetEmployeesFiltered
                            @DeptID INT,
                            @MinSalary INT,
                            @MaxSalary INT
                        AS
                        BEGIN
                            SELECT EmployeeID, Name, Salary
                            FROM Employees
                            WHERE DepartmentID = @DeptID 
                            AND Salary BETWEEN @MinSalary AND @MaxSalary;
                        END;

Calling the Procedure

                        sqlCopyEditEXEC GetEmployeesFiltered @DeptID = 101, @MinSalary = 5000, @MaxSalary = 10000;

💡 Benefits:
✔️ Reusable Query Logic
✔️ Better Performance vs Dynamic Queries

---

2.3 Output Parameters in Stored Procedures

Stored procedures can return values using OUTPUT parameters.

💡 Use Case: Get Employee Count in a Department

                        sqlCopyEditCREATE PROCEDURE GetEmployeeCount
                            @DeptID INT,
                            @EmpCount INT OUTPUT
                        AS
                        BEGIN
                            SELECT @EmpCount = COUNT(*) 
                            FROM Employees
                            WHERE DepartmentID = @DeptID;
                        END;

Calling the Procedure

                        sqlCopyEditDECLARE @TotalEmployees INT;
                        EXEC GetEmployeeCount @DeptID = 101, @EmpCount = @TotalEmployees OUTPUT;
                        PRINT 'Total Employees: ' + CAST(@TotalEmployees AS NVARCHAR);

👉 Output: Total Employees: 5

💡 Key Benefits:
✔️ Output values directly to calling programs
✔️ Used in complex calculations and reporting

---

2.4 Stored Procedure with TRY...CATCH for Error Handling

💡 Use Case: Handle Errors in Transactions

                        sqlCopyEditCREATE PROCEDURE TransferFunds
                            @FromAccount INT,
                            @ToAccount INT,
                            @Amount DECIMAL(10,2)
                        AS
                        BEGIN
                            BEGIN TRANSACTION;

                            BEGIN TRY
                                UPDATE Accounts SET Balance = Balance - @Amount WHERE AccountID = @FromAccount;
                                UPDATE Accounts SET Balance = Balance + @Amount WHERE AccountID = @ToAccount;

                                COMMIT TRANSACTION;
                            END TRY
                            BEGIN CATCH
                                ROLLBACK TRANSACTION;
                                PRINT 'Error Occurred: ' + ERROR_MESSAGE();
                            END CATCH;
                        END;

Calling the Procedure

                        sqlCopyEditEXEC TransferFunds @FromAccount = 1, @ToAccount = 2, @Amount = 500;

💡 Key Benefits:
✔️ Ensures Data Consistency
✔️ Prevents Half-Completed Transactions

---

3. When to Use Functions vs Stored Procedures?

---

4. Best Practices for Functions & Stored Procedures

✅ Use Inline TVFs when possible for best performance.
✅ Use Scalar Functions for calculations, but avoid inside loops (performance issue).
✅ Use Stored Procedures for complex business logic and transactions.
✅ Always Use TRY...CATCH for Error Handling in Procedures.
✅ Use Output Parameters in Stored Procedures for Returning Values.

---

Final Thoughts

✅ Scalar Functions return a single value, Table-Valued Functions return a table.
✅ Inline TVFs perform better than Multi-Statement TVFs.
✅ Stored Procedures handle complex business logic & transactions.
✅ Use Output Parameters in Stored Procedures for returning values efficiently.

🔎 Want More SQL Performance Tips?
👉 Follow me on Hashnode for real-world SQL challenges, optimizations, and best practices! 🚀

---

• MERGE Statement for UPSERT Operations

• ChatGPT said:

  Mastering the MERGE Statement for UPSERT Operations in SQL Server 🚀

  1. Introduction to MERGE in SQL Server

  The MERGE statement in T-SQL allows performing INSERT, UPDATE, and DELETE operations in a single statement, making it ideal for UPSERT (Update + Insert) operations.

  💡 Why Use MERGE?
  ✔️ Efficient UPSERT operation (Update existing records, Insert new ones)
  ✔️ Reduces multiple queries into one atomic operation
  ✔️ Maintains data integrity & improves performance
  ✔️ Supports DELETE when records no longer exist

  ---

  2. Basic Syntax of MERGE

    sqlCopyEditMERGE TargetTable AS T
    USING SourceTable AS S
    ON T.KeyColumn = S.KeyColumn
    WHEN MATCHED THEN
        UPDATE SET T.Column1 = S.Column1
    WHEN NOT MATCHED THEN
        INSERT (Column1, Column2) VALUES (S.Column1, S.Column2);
  

  🔹 Explanation
  ✔️ USING → Defines the source (new data).
  ✔️ ON → Defines the matching condition (common key).
  ✔️ WHEN MATCHED → Updates existing records.
  ✔️ WHEN NOT MATCHED → Inserts new records.

  ---

  3. Implementing an UPSERT with MERGE

  💡 Use Case: Syncing Employee Data

    sqlCopyEditMERGE INTO Employees AS Target
    USING EmployeeUpdates AS Source
    ON Target.EmployeeID = Source.EmployeeID
  
    WHEN MATCHED THEN 
        UPDATE SET Target.Name = Source.Name, Target.Salary = Source.Salary
  
    WHEN NOT MATCHED THEN 
        INSERT (EmployeeID, Name, Salary)
        VALUES (Source.EmployeeID, Source.Name, Source.Salary);
  

  🔹 How It Works?
  ✔️ If EmployeeID exists, it updates Name & Salary.
  ✔️ If EmployeeID does not exist, it inserts a new record.

  ---

  4. Handling Deletions with MERGE

  💡 Use Case: Remove Employees No Longer in Source

    sqlCopyEditMERGE INTO Employees AS Target
    USING EmployeeUpdates AS Source
    ON Target.EmployeeID = Source.EmployeeID
  
    WHEN MATCHED THEN 
        UPDATE SET Target.Name = Source.Name, Target.Salary = Source.Salary
  
    WHEN NOT MATCHED BY SOURCE THEN 
        DELETE;  -- Deletes records that are missing in Source
  

  ✔️ Deletes employees from Employees if they no longer exist in EmployeeUpdates.

  ---

  5. Using OUTPUT with MERGE

  🔹 OUTPUT returns details of affected rows (Inserted, Updated, Deleted).

  💡 Use Case: Track Changes During MERGE

    sqlCopyEditMERGE INTO Employees AS Target
    USING EmployeeUpdates AS Source
    ON Target.EmployeeID = Source.EmployeeID
  
    WHEN MATCHED THEN 
        UPDATE SET Target.Name = Source.Name, Target.Salary = Source.Salary
  
    WHEN NOT MATCHED THEN 
        INSERT (EmployeeID, Name, Salary)
        VALUES (Source.EmployeeID, Source.Name, Source.Salary)
  
    WHEN NOT MATCHED BY SOURCE THEN 
        DELETE
  
    OUTPUT $ACTION, inserted.*, deleted.*;
  

  👉 Output Example

  | $ACTION | EmployeeID | Name | Salary | Old Name | Old Salary | | --- | --- | --- | --- | --- | --- | | INSERT | 5 | Sam | 6000 | NULL | NULL | | UPDATE | 3 | Jane | 7500 | Jane | 7000 | | DELETE | 7 | NULL | NULL | Bob | 5500 |

  💡 $ACTION Values
  ✔️ INSERT – New record added
  ✔️ UPDATE – Existing record modified
  ✔️ DELETE – Record removed

  ---

  6. Using MERGE with Transactions for Data Consistency

  💡 Use Case: Ensuring Atomicity in UPSERT

    sqlCopyEditBEGIN TRANSACTION;
  
    BEGIN TRY
        MERGE INTO Employees AS Target
        USING EmployeeUpdates AS Source
        ON Target.EmployeeID = Source.EmployeeID
  
        WHEN MATCHED THEN 
            UPDATE SET Target.Name = Source.Name, Target.Salary = Source.Salary
  
        WHEN NOT MATCHED THEN 
            INSERT (EmployeeID, Name, Salary)
            VALUES (Source.EmployeeID, Source.Name, Source.Salary)
  
        WHEN NOT MATCHED BY SOURCE THEN 
            DELETE;
  
        COMMIT TRANSACTION;
    END TRY
    BEGIN CATCH
        ROLLBACK TRANSACTION;
        PRINT 'Error Occurred: ' + ERROR_MESSAGE();
    END CATCH;
  

  🔹 Why Use a Transaction?
  ✔️ Ensures all operations complete successfully
  ✔️ Rollback on failure prevents partial updates

  ---

  7. Using MERGE for Slowly Changing Dimensions (SCD)

  🔹 SCD Type 2: Maintain history while updating new records.

    sqlCopyEditMERGE INTO Customers AS Target
    USING CustomerUpdates AS Source
    ON Target.CustomerID = Source.CustomerID
  
    WHEN MATCHED AND Target.Address <> Source.Address THEN
        UPDATE SET Target.EndDate = GETDATE()  -- Marks old record as inactive
  
    WHEN NOT MATCHED THEN 
        INSERT (CustomerID, Name, Address, StartDate)
        VALUES (Source.CustomerID, Source.Name, Source.Address, GETDATE());
  

  ✔️ Keeps old records while inserting updates
  ✔️ Useful in data warehouses & analytics

  ---

  8. Performance Considerations for MERGE

  | Consideration | Best Practice | | --- | --- | | Indexing | Ensure indexed ON conditions for better performance | | Batch Processing | Process large data in smaller batches | | Locking | Use WITH (ROWLOCK) to prevent table locks | | Transaction Management | Wrap MERGE in transactions to ensure rollback |

  ---

  9. Common Errors and Fixes

  | Error | Cause | Fix | | --- | --- | --- | | Duplicate Key Error | MERGE attempts to insert a duplicate primary key | Ensure source data has no duplicates | | Deadlocks | Multiple sessions running MERGE | Use ROWLOCK or run queries in batches | | Incorrect Deletes | WHEN NOT MATCHED BY SOURCE THEN DELETE removes unintended rows | Add extra conditions for deletion |

  ---

  10. When to Use MERGE vs Traditional Queries

  | Feature | MERGE | Traditional Queries (IF EXISTS + UPDATE/INSERT) | | --- | --- | --- | | Performance | ✅ Faster for bulk operations | ❌ Slower for large datasets | | Readability | ✅ Compact & easy to read | ❌ Requires multiple statements | | Flexibility | ✅ Supports INSERT, UPDATE, DELETE in one query | ❌ Requires multiple queries |

  ---

  Final Thoughts

  ✅ MERGE efficiently performs UPSERT (Insert + Update) operations
  ✅ Use WHEN NOT MATCHED BY SOURCE THEN DELETE carefully
  ✅ Leverage OUTPUT $ACTION to track changes
  ✅ Use transactions (BEGIN TRANSACTION) for safe execution
  ✅ Ensure indexes on ON conditions for performance boost

  🔎 Want More SQL Performance Tips?
  👉 Follow me on Hashnode for real-world SQL challenges, optimizations, and advanced analytics! 🚀

Advanced Data Manipulation in SQL Server 🚀

3.1 Introduction to Advanced Data Manipulation

SQL Server provides powerful tools for data manipulation, transformation, and security. This guide covers:
✔️ Advanced INSERT, UPDATE, DELETE with OUTPUT Clause
✔️ Data Type Conversion & Parsing (CAST, CONVERT, TRY_CAST)
✔️ Handling JSON & XML Data
✔️ Temporal Tables for Historical Data Management
✔️ Data Encryption & Decryption

---

3.2 Advanced INSERT, UPDATE, DELETE with OUTPUT Clause

What is the OUTPUT Clause?

The OUTPUT clause captures affected rows during INSERT, UPDATE, and DELETE operations.

💡 Use Case: Capture Inserted IDs

                        sqlCopyEditDECLARE @InsertedData TABLE (EmployeeID INT, Name NVARCHAR(100));

                        INSERT INTO Employees (Name, Salary)
                        OUTPUT INSERTED.EmployeeID, INSERTED.Name INTO @InsertedData
                        VALUES ('John Doe', 7500);

                        SELECT * FROM @InsertedData;

👉 Captured Output:

---

💡 Use Case: Track Updated Values

                        sqlCopyEditDECLARE @UpdatedData TABLE (OldSalary INT, NewSalary INT);

                        UPDATE Employees
                        SET Salary = Salary * 1.1
                        OUTPUT DELETED.Salary AS OldSalary, INSERTED.Salary AS NewSalary
                        INTO @UpdatedData;

                        SELECT * FROM @UpdatedData;

👉 Captured Output:

---

💡 Use Case: Track Deleted Records

                        sqlCopyEditDECLARE @DeletedData TABLE (EmployeeID INT, Name NVARCHAR(100));

                        DELETE FROM Employees
                        OUTPUT DELETED.EmployeeID, DELETED.Name INTO @DeletedData
                        WHERE Salary < 5000;

                        SELECT * FROM @DeletedData;

👉 Captured Output:

💡 Why Use OUTPUT?
✔️ Captures affected rows for logging & auditing
✔️ Useful for undo operations & history tracking

---

3.3 Data Type Conversion and Parsing

🔹 SQL Server provides multiple functions for data conversion:
✔️ CAST() – Converts data explicitly
✔️ CONVERT() – Converts with format options
✔️ TRY_CAST(), TRY_CONVERT() – Returns NULL instead of errors

---

💡 CAST() Example: Convert Salary to NVARCHAR

                        sqlCopyEditSELECT CAST(Salary AS NVARCHAR) AS SalaryString FROM Employees;

👉 Output: '7500', '8000'

---

💡 CONVERT() Example: Convert Date to String

                        sqlCopyEditSELECT CONVERT(VARCHAR, GETDATE(), 103) AS FormattedDate;

👉 Output: 12/02/2024 (UK format)

🔹 Common Date Formats in CONVERT()

---

💡 TRY_CAST() and TRY_CONVERT() (Safe Conversions)

                        sqlCopyEditSELECT TRY_CAST('1234' AS INT) AS SafeConvert1,
                               TRY_CAST('ABC' AS INT) AS SafeConvert2;

👉 Output:

💡 Why Use TRY_CAST()?
✔️ Prevents conversion errors (returns NULL on failure).
✔️ Useful for parsing user input safely.

---

3.4 JSON and XML Data Handling in SQL Server

Working with JSON Data

SQL Server natively supports JSON for storing & querying structured data.

💡 Use Case: Convert Table Data to JSON

                        sqlCopyEditSELECT EmployeeID, Name, Salary
                        FROM Employees
                        FOR JSON AUTO;

👉 Output:

                        jsonCopyEdit[{"EmployeeID":101,"Name":"John","Salary":7500},
                         {"EmployeeID":102,"Name":"Jane","Salary":8000}]

---

💡 Extract Values from JSON

                        sqlCopyEditDECLARE @json NVARCHAR(MAX) = '{"EmployeeID":101,"Name":"John","Salary":7500}';

                        SELECT JSON_VALUE(@json, '$.Name') AS EmployeeName;

👉 Output: John

💡 Key Functions for JSON Handling
✔️ JSON_VALUE() – Extracts single values
✔️ OPENJSON() – Converts JSON to table format

---

Working with XML Data

💡 Convert Table Data to XML

                        sqlCopyEditSELECT EmployeeID, Name, Salary
                        FROM Employees
                        FOR XML AUTO;

👉 Output:

                        xmlCopyEdit<Employees EmployeeID="101" Name="John" Salary="7500" />
                        <Employees EmployeeID="102" Name="Jane" Salary="8000" />

---

💡 Parse XML Data

                        sqlCopyEditDECLARE @xml XML = '<Employee><ID>101</ID><Name>John</Name></Employee>';

                        SELECT @xml.value('(/Employee/Name)[1]', 'NVARCHAR(50)') AS EmployeeName;

👉 Output: John

💡 Key Functions for XML Handling
✔️ FOR XML AUTO – Convert data to XML
✔️ .value() – Extract values from XML

---

3.5 Temporal Tables and Historical Data Management

Temporal tables automatically track data changes.

💡 Enable Temporal Table

                        sqlCopyEditCREATE TABLE EmployeeHistory (
                            EmployeeID INT PRIMARY KEY,
                            Name NVARCHAR(100),
                            Salary INT,
                            ValidFrom DATETIME2 GENERATED ALWAYS AS ROW START,
                            ValidTo DATETIME2 GENERATED ALWAYS AS ROW END,
                            PERIOD FOR SYSTEM_TIME (ValidFrom, ValidTo)
                        ) WITH (SYSTEM_VERSIONING = ON);

💡 Why Use Temporal Tables?
✔️ Automatically stores history
✔️ Allows time-travel queries

---

3.6 Data Encryption and Decryption in SQL Server

Column-Level Encryption

💡 Encrypt Data Using ENCRYPTBYPASSPHRASE()

                        sqlCopyEditDECLARE @Encrypted VARBINARY(MAX);
                        SET @Encrypted = ENCRYPTBYPASSPHRASE('MyKey', 'SensitiveData');

                        SELECT @Encrypted AS EncryptedData;

---

💡 Decrypt Data Using DECRYPTBYPASSPHRASE()

                        sqlCopyEditSELECT CAST(DECRYPTBYPASSPHRASE('MyKey', @Encrypted) AS NVARCHAR);

💡 Why Use Encryption?
✔️ Protects sensitive information (e.g., passwords, credit card details).
✔️ Prevents unauthorized access.

---

Transparent Data Encryption (TDE)

✔️ Encrypts entire database at rest
✔️ Protects backups from theft

                        sqlCopyEditALTER DATABASE MyDatabase SET ENCRYPTION ON;

🔹 TDE Automatically Handles Encryption/Decryption

---

Final Thoughts

✅ Use OUTPUT Clause for tracking INSERT, UPDATE, DELETE changes.
✅ Use TRY_CAST() for safe data conversions.
✅ Use JSON and XML for structured data storage & queries.
✅ Use Temporal Tables for historical data tracking.
✅ Use Data Encryption (TDE, ENCRYPTBYPASSPHRASE) for security.

🔎 Want More SQL Performance Tips?
👉 Follow me on Hashnode for real-world SQL challenges, optimizations, and best practices! 🚀

---

1. SQL Server Query Optimization Techniques 🚀

   4.1 Query Execution Plans (Estimated vs. Actual)

   What is a Query Execution Plan?

   A Query Execution Plan is a roadmap that SQL Server uses to retrieve or modify data. It shows:
   ✔️ Indexes Used
   ✔️ Joins & Operations
   ✔️ Estimated vs. Actual Row Counts
   ✔️ Performance Bottlenecks

   ---

   💡 How to View Execution Plans?

   1️⃣ Estimated Execution Plan (Before Running Query)

    sqlCopyEditSET SHOWPLAN_XML ON;
    SELECT * FROM Employees WHERE DepartmentID = 101;
    SET SHOWPLAN_XML OFF;
   

   ✔️ Does NOT execute the query
   ✔️ Shows the expected execution plan

   2️⃣ Actual Execution Plan (After Running Query)

    sqlCopyEditSET STATISTICS XML ON;
    SELECT * FROM Employees WHERE DepartmentID = 101;
    SET STATISTICS XML OFF;
   

   ✔️ Executes the query
   ✔️ Shows real-time performance

   👉 Best Practice: Use Estimated Plan for tuning and Actual Plan for troubleshooting.

   ---

   4.2 Index Usage and Tuning

   Indexes speed up query performance by organizing data efficiently.

   Types of Indexes

   | Index Type | Description | Use Case | | --- | --- | --- | | Clustered | Sorts & stores data physically | Primary Key (1 per table) | | Non-Clustered | Stores pointers to data | Frequently searched columns | | Unique | Enforces unique values | Emails, SSNs, etc. | | Filtered | Indexes specific rows | Sparse Data (NULLs, Partial Data) |

   ---

   💡 Use Case: Creating Clustered & Non-Clustered Indexes

    sqlCopyEdit-- Clustered Index (Only One Per Table)
    CREATE CLUSTERED INDEX IX_Employees ON Employees(EmployeeID);
   
    -- Non-Clustered Index (For Faster Lookups)
    CREATE NONCLUSTERED INDEX IX_Employees_Department ON Employees(DepartmentID);
   

   🔹 How It Helps?
   ✔️ Clustered Index → Improves sorting & retrieval.
   ✔️ Non-Clustered Index → Speeds up WHERE & JOIN operations.

   ---

   4.3 Query Hints and Optimizer Hints

   🔹 SQL Server Query Optimizer decides the best execution plan, but hints allow manual tuning.

   💡 Use Case: Forcing an Index

    sqlCopyEditSELECT * FROM Employees WITH (INDEX(IX_Employees_Department))
    WHERE DepartmentID = 101;
   

   ✔️ Forces Non-Clustered Index Usage

   ---

   Common Query Hints

   | Query Hint | Description | | --- | --- | | OPTION (RECOMPILE) | Avoids query plan reuse | | WITH (NOLOCK) | Reads data without locking (Dirty Reads) | | FORCESEEK | Forces an index seek instead of scan |

   💡 Use Case: Preventing Locking Issues

    sqlCopyEditSELECT * FROM Orders WITH (NOLOCK);
   

   ✔️ Prevents locks but allows dirty reads

   👉 Best Practice: Use hints only when necessary, as they override SQL Server’s optimization.

   ---

   4.4 Table Variables vs. Temporary Tables vs. CTEs

   | Feature | Table Variable (@TableVar) | Temporary Table (#TempTable) | CTE (WITH CTE AS) | | --- | --- | --- | --- | | Scope | Batch/Session | Session/Connection | Query Scope | | Indexing | No Indexes | Allows Indexing | No Indexes | | Performance | Fast for small data | Better for large data | Optimized for recursive queries | | Use Case | Small datasets, Stored Procedures | Large datasets, Complex Joins | Recursive Queries |

   ---

   💡 Use Case: When to Use Each

   1️⃣ Table Variable (@TableVar) - Small Datasets

    sqlCopyEditDECLARE @TempTable TABLE (EmployeeID INT, Name NVARCHAR(100));
    INSERT INTO @TempTable VALUES (1, 'Alice'), (2, 'Bob');
    SELECT * FROM @TempTable;
   

   2️⃣ Temporary Table (#TempTable) - Large Datasets

    sqlCopyEditCREATE TABLE #TempTable (EmployeeID INT, Name NVARCHAR(100));
    INSERT INTO #TempTable SELECT EmployeeID, Name FROM Employees;
    SELECT * FROM #TempTable;
    DROP TABLE #TempTable;
   

   3️⃣ CTE - Recursive Queries

    sqlCopyEditWITH EmployeeHierarchy AS (
        SELECT EmployeeID, Name, ManagerID FROM Employees WHERE ManagerID IS NULL
        UNION ALL
        SELECT e.EmployeeID, e.Name, e.ManagerID FROM Employees e
        INNER JOIN EmployeeHierarchy eh ON e.ManagerID = eh.EmployeeID
    )
    SELECT * FROM EmployeeHierarchy;
   

   💡 Best Practice:
   ✔️ Use Table Variables for small datasets.
   ✔️ Use Temporary Tables for complex joins.
   ✔️ Use CTEs for hierarchical queries.

   ---

   4.5 Batch Processing and Bulk Data Operations

   🔹 Batch Processing helps optimize large inserts, updates, or deletes.

   💡 Use Case: Batch Processing for Large Inserts

    sqlCopyEditDECLARE @BatchSize INT = 1000;
    DECLARE @Counter INT = 1;
   
    WHILE @Counter <= (SELECT COUNT(*) FROM NewEmployees)
    BEGIN
        INSERT INTO Employees
        SELECT TOP (@BatchSize) * FROM NewEmployees
        ORDER BY EmployeeID;
   
        SET @Counter = @Counter + @BatchSize;
    END;
   

   🔹 Why Use Batch Processing?
   ✔️ Prevents log file bloating
   ✔️ Reduces locks & contention

   ---

   💡 BULK INSERT for Fast Data Loading

    sqlCopyEditBULK INSERT Employees
    FROM 'C:\Data\Employees.csv'
    WITH (FORMAT = 'CSV', FIRSTROW = 2);
   

   🔹 Why Use BULK INSERT?
   ✔️ 100x Faster than INSERT INTO
   ✔️ Supports CSV, Flat Files

   ---

   4.6 Managing Query Performance with Statistics

   🔹 SQL Server uses statistics to estimate row counts & optimize queries.

   Check Statistics

    sqlCopyEditDBCC SHOW_STATISTICS ('Employees', IX_Employees_Department);
   

   ---

   💡 Update Statistics for Performance Optimization

    sqlCopyEditUPDATE STATISTICS Employees;
   

   🔹 Best Practice:
   ✔️ Regularly update statistics to ensure accurate execution plans.

   ---

   Final SQL Performance Optimization Cheat Sheet

   | Technique | Benefit | | --- | --- | | Indexes (Clustered/Non-Clustered) | Speed up lookups | | Execution Plans | Identify slow queries | | Query Hints (NOLOCK, FORCESEEK) | Tune optimizer behavior | | Batch Processing | Reduce locks on large operations | | CTEs & Temp Tables | Improve recursive & complex queries | | Statistics Updates | Ensure accurate execution plans |

   ---

   Final Thoughts

   ✅ Execution Plans help optimize queries before execution.
   ✅ Indexes improve query speed, but improper indexing can slow performance.
   ✅ Use Query Hints selectively to fine-tune SQL Server performance.
   ✅ Batch Processing prevents locking issues for bulk operations.
   ✅ Regularly update statistics to ensure accurate query execution plans.

   🔎 Want More SQL Performance Tips?
   👉 Follow me on Hashnode for real-world SQL optimizations & database tuning! 🚀

   ---

Advanced Indexing and Partitioning in SQL Server 🚀

5.1 Index Design for High Performance

Indexes optimize query performance by allowing SQL Server to retrieve data efficiently.

💡 Key Considerations for High-Performance Indexing

✔️ Use Clustered Index for sorting & faster lookups
✔️ Use Non-Clustered Index on frequently queried columns
✔️ Use Unique Indexes for ensuring data integrity
✔️ Use Filtered Indexes to optimize sparse data queries
✔️ Avoid too many indexes (hurts INSERT, UPDATE, DELETE)

---

💡 Creating Clustered and Non-Clustered Indexes

            sqlCopyEdit-- Create a Clustered Index on the Primary Key
            CREATE CLUSTERED INDEX IX_Employees ON Employees(EmployeeID);

            -- Create a Non-Clustered Index on DepartmentID for faster lookups
            CREATE NONCLUSTERED INDEX IX_Employees_Department ON Employees(DepartmentID);

🔹 Why This Works?
✔️ Clustered Index physically sorts data
✔️ Non-Clustered Index speeds up searches on DepartmentID

---

💡 Creating a Filtered Index (Optimize Sparse Data)

            sqlCopyEditCREATE NONCLUSTERED INDEX IX_Employees_Active ON Employees(Status)
            WHERE Status = 'Active';

🔹 Why Use Filtered Indexes?
✔️ Reduces index size (uses less storage)
✔️ Optimizes queries on specific subsets of data

---

5.2 Indexed Views

What is an Indexed View?

🔹 An Indexed View is a materialized view stored physically on disk.

✔️ Improves performance of complex aggregations
✔️ Stores precomputed results for fast retrieval
✔️ Requires SCHEMABINDING

---

💡 Creating an Indexed View

            sqlCopyEditCREATE VIEW SalesSummary WITH SCHEMABINDING AS
            SELECT DepartmentID, COUNT(*) AS EmployeeCount, AVG(Salary) AS AvgSalary
            FROM dbo.Employees
            GROUP BY DepartmentID;

            -- Create a Unique Clustered Index on the View
            CREATE UNIQUE CLUSTERED INDEX IX_SalesSummary ON SalesSummary(DepartmentID);

🔹 Why Use Indexed Views?
✔️ Precomputes aggregations for faster queries
✔️ Improves performance for reporting

---

5.3 Columnstore Indexes and Data Compression

🔹 Columnstore Indexes store data column-wise instead of row-wise, enabling fast analytics.
✔️ Best for large analytical queries (OLAP)
✔️ Reduces I/O and improves compression

---

💡 Creating a Columnstore Index

            sqlCopyEditCREATE CLUSTERED COLUMNSTORE INDEX IX_Employee_Columnstore
            ON Employees;

🔹 Why Columnstore?
✔️ Great for analytics (e.g., SUM, AVG)
✔️ Compressed storage = lower disk usage

---

💡 Using Page & Row Compression

            sqlCopyEdit-- Enable Page Compression (Best for OLAP)
            ALTER TABLE Employees REBUILD WITH (DATA_COMPRESSION = PAGE);

            -- Enable Row Compression (Best for OLTP)
            ALTER TABLE Employees REBUILD WITH (DATA_COMPRESSION = ROW);

🔹 Why Use Compression?
✔️ Saves storage
✔️ Speeds up reads by reducing I/O

---

5.4 Table Partitioning and Sliding Window Techniques

What is Table Partitioning?

🔹 Splits large tables into smaller partitions for faster query performance.

✔️ Improves query performance for large datasets
✔️ Reduces locking & contention
✔️ Helps with data archiving strategies

---

💡 Step 1: Create a Partition Function

            sqlCopyEditCREATE PARTITION FUNCTION pf_Sales (DATE)
            AS RANGE RIGHT FOR VALUES ('2023-01-01', '2023-07-01', '2024-01-01');

✔️ Splits data into four partitions
✔️ Ranges are exclusive on the left

---

💡 Step 2: Create a Partition Scheme

            sqlCopyEditCREATE PARTITION SCHEME ps_Sales
            AS PARTITION pf_Sales ALL TO ([PRIMARY]);

✔️ Maps partitions to storage locations

---

💡 Step 3: Create a Partitioned Table

            sqlCopyEditCREATE TABLE Sales (
                SaleID INT PRIMARY KEY,
                SaleDate DATE NOT NULL,
                Amount DECIMAL(10,2)
            ) ON ps_Sales(SaleDate);

✔️ Automatically partitions data based on SaleDate

---

Sliding Window Technique (Efficient Archiving)

            sqlCopyEdit-- Step 1: Switch Old Data to an Archive Table
            ALTER TABLE Sales SWITCH PARTITION 1 TO Sales_Archive;

            -- Step 2: Drop Old Partition
            ALTER PARTITION FUNCTION pf_Sales() MERGE RANGE ('2023-01-01');

            -- Step 3: Add New Future Partition
            ALTER PARTITION FUNCTION pf_Sales() SPLIT RANGE ('2025-01-01');

🔹 Why Use Sliding Windows?
✔️ Efficiently move old data to archive
✔️ Keeps the table size optimized

---

5.5 Querying Partitioned Tables

💡 Optimized Query Using Partitioning

            sqlCopyEditSELECT * FROM Sales
            WHERE SaleDate >= '2024-01-01' AND SaleDate < '2024-07-01';

🔹 Partitioning Benefits ✔️ SQL Server automatically eliminates unnecessary partitions
✔️ Query performance improves significantly

---

💡 Checking Which Partition is Used

            sqlCopyEditSELECT $PARTITION.pf_Sales(SaleDate) AS PartitionUsed, *
            FROM Sales WHERE SaleDate = '2024-05-15';

✔️ Identifies which partition stores the data

---

💡 Splitting and Merging Partitions for Maintenance

            sqlCopyEdit-- Merge an Old Partition (Combine Two Ranges)
            ALTER PARTITION FUNCTION pf_Sales() MERGE RANGE ('2023-07-01');

            -- Split to Add a Future Partition
            ALTER PARTITION FUNCTION pf_Sales() SPLIT RANGE ('2025-01-01');

🔹 Why Use Partition Management?
✔️ Merges historical data
✔️ Adds future partitions without downtime

---

Final Thoughts

✅ Indexes optimize performance, but use wisely to balance INSERT/UPDATE/DELETE impact.
✅ Indexed Views precompute aggregations for high-speed reporting.
✅ Columnstore Indexes & Data Compression improve large-scale analytics.
✅ Table Partitioning manages large datasets efficiently.
✅ Sliding Window Partitioning automates data archiving & improves query speed.

🔎 Want More SQL Performance Tips?
👉 Follow me on Hashnode for real-world SQL optimizations & database tuning! 🚀

---

1. SQL Server Security and Permissions 🔒

   SQL Server offers multiple layers of security to protect data from unauthorized access. This guide covers:
   ✔️ Role-Based Access Control (RBAC)
   ✔️ GRANT, REVOKE, DENY Statements
   ✔️ Row-Level Security (RLS)
   ✔️ Dynamic Data Masking (DDM)
   ✔️ Transparent Data Encryption (TDE)

   ---

   6.1 Role-Based Access Control (RBAC)

   What is RBAC?

   🔹 Role-Based Access Control (RBAC) allows grouping users into roles and granting permissions to roles instead of individual users.

   ✔️ Simplifies security management
   ✔️ Minimizes direct user permissions
   ✔️ Ensures better security compliance

   ---

   💡 Creating and Assigning Server Roles

    sqlCopyEdit-- Create a Custom Server Role
    CREATE SERVER ROLE DataAnalyst;
   
    -- Add a User to the Role
    ALTER SERVER ROLE DataAnalyst ADD MEMBER JohnDoe;
   

   ✔️ Grants role-based access at the SERVER level.

   ---

   💡 Creating and Assigning Database Roles

    sqlCopyEdit-- Create a Custom Database Role
    CREATE ROLE SalesManager;
   
    -- Grant SELECT & UPDATE Permissions to the Role
    GRANT SELECT, UPDATE ON Sales TO SalesManager;
   
    -- Add a User to the Role
    ALTER ROLE SalesManager ADD MEMBER JaneDoe;
   

   ✔️ Users assigned to SalesManager can SELECT & UPDATE Sales data.

   ---

   6.2 GRANT, REVOKE, DENY Statements

   🔹 Permissions in SQL Server are managed using:
   ✔️ GRANT – Gives permission
   ✔️ REVOKE – Removes granted permission
   ✔️ DENY – Explicitly blocks access

   ---

   💡 Granting Permissions

    sqlCopyEditGRANT SELECT ON Employees TO JohnDoe;
    GRANT INSERT, UPDATE ON Orders TO SalesManager;
   

   ✔️ JohnDoe can now read Employees
   ✔️ SalesManager can insert & update Orders

   ---

   💡 Revoking Permissions

    sqlCopyEditREVOKE UPDATE ON Orders FROM SalesManager;
   

   ✔️ Removes UPDATE access but keeps other permissions.

   ---

   💡 Denying Permissions

    sqlCopyEditDENY DELETE ON Employees TO JohnDoe;
   

   ✔️ JohnDoe CANNOT delete Employees, even if part of another role.

   ---

   6.3 Row-Level Security (RLS)

   🔹 RLS restricts access at the row level, ensuring users see only their allowed data.

   ✔️ Best for multi-tenant applications
   ✔️ Prevents unauthorized data exposure
   ✔️ Implemented using Security Policies

   ---

   💡 Step 1: Create a Security Predicate Function

    sqlCopyEditCREATE FUNCTION dbo.EmployeeSecurityPredicate(@EmployeeID INT)
    RETURNS TABLE
    WITH SCHEMABINDING
    AS
    RETURN SELECT 1 AS AccessAllowed WHERE @EmployeeID = USER_ID();
   

   ✔️ Ensures users can only see their EmployeeID data.

   ---

   💡 Step 2: Apply Row-Level Security Policy

    sqlCopyEditCREATE SECURITY POLICY EmployeeSecurityPolicy
    ADD FILTER PREDICATE dbo.EmployeeSecurityPredicate(EmployeeID)
    ON Employees;
   

   ✔️ Automatically restricts query results based on EmployeeID.

   ---

   💡 Testing RLS Policy

    sqlCopyEditEXECUTE AS USER = 'JohnDoe';
    SELECT * FROM Employees; -- Only John's data is visible
    REVERT;
   

   ✔️ Users only see rows they are authorized to access.

   ---

   6.4 Dynamic Data Masking (DDM)

   🔹 DDM masks sensitive data to prevent unauthorized viewing.

   ✔️ Protects sensitive data like SSN, Credit Cards
   ✔️ Easy implementation, No changes in application code
   ✔️ Users with UNMASK permission can view real data

   ---

   💡 Adding Dynamic Data Masking

    sqlCopyEditALTER TABLE Customers
    ALTER COLUMN SSN ADD MASKED WITH (FUNCTION = 'partial(2,"XXX-XX-",4)');
   

   ✔️ Masks SSN as 12X-XX-1234 while keeping the first 2 and last 4 digits visible.

   ---

   💡 Viewing Masked Data

    sqlCopyEditSELECT SSN FROM Customers;
   

   👉 Output (Masked Data)

    CopyEdit12X-XX-1234
    45X-XX-5678
   

   ✔️ Users without UNMASK permission see masked values.

   ---

   💡 Granting UNMASK Permission

    sqlCopyEditGRANT UNMASK TO DataAuditor;
   

   ✔️ DataAuditor can see the actual SSNs.

   ---

   6.5 Transparent Data Encryption (TDE)

   🔹 TDE encrypts the entire database at rest to protect against unauthorized access.

   ✔️ Encrypts the database, backups, and logs
   ✔️ Transparent to applications (No changes needed)
   ✔️ Prevents unauthorized access to database files

   ---

   💡 Step 1: Create a Database Master Key

    sqlCopyEditCREATE MASTER KEY ENCRYPTION BY PASSWORD = 'StrongPass@123!';
   

   ---

   💡 Step 2: Create a Certificate for Encryption

    sqlCopyEditCREATE CERTIFICATE TDE_Cert WITH SUBJECT = 'Database Encryption Key';
   

   ---

   💡 Step 3: Create a Database Encryption Key

    sqlCopyEditUSE MyDatabase;
    CREATE DATABASE ENCRYPTION KEY
    WITH ALGORITHM = AES_256
    ENCRYPTION BY SERVER CERTIFICATE TDE_Cert;
   

   ---

   💡 Step 4: Enable Transparent Data Encryption

    sqlCopyEditALTER DATABASE MyDatabase SET ENCRYPTION ON;
   

   ✔️ Database is now encrypted, including backups.

   ---

   💡 Verify TDE Status

    sqlCopyEditSELECT name, is_encrypted FROM sys.databases WHERE name = 'MyDatabase';
   

   👉 Output:

   | Database | is_encrypted | | --- | --- | | MyDatabase | 1 |

   ✔️ 1 = Encrypted, 0 = Not Encrypted

   ---

   🔐 Final SQL Security Best Practices

   | Security Feature | Purpose | | --- | --- | | RBAC (Roles & Permissions) | Manage user access efficiently | | GRANT, REVOKE, DENY | Fine-grained permission control | | Row-Level Security (RLS) | Restrict access at row level | | Dynamic Data Masking (DDM) | Protects sensitive data | | Transparent Data Encryption (TDE) | Encrypts database at rest |

   ---

   Final Thoughts

   ✅ RBAC simplifies permission management.
   ✅ GRANT, REVOKE, DENY fine-tune database security.
   ✅ Row-Level Security ensures users see only their data.
   ✅ Dynamic Data Masking prevents unauthorized data exposure.
   ✅ TDE encrypts the entire database for data security.

   🔎 Want More SQL Security Tips?
   👉 Follow me on Hashnode for real-world SQL security techniques! 🚀

   ---

   Working with Advanced Data Types in SQL Server 🚀

   SQL Server supports advanced data types for geospatial data, JSON, XML, and user-defined types. These help in handling structured, semi-structured, and spatial data efficiently.

   ---

   7.1 Spatial Data Types and Queries 🌍

   🔹 SQL Server supports spatial data types (GEOMETRY and GEOGRAPHY) for storing and querying spatial data.

   ✔️ GEOMETRY – Stores planar (flat-earth) data
   ✔️ GEOGRAPHY – Stores geodetic (spherical-earth) data

   ---

   💡 Creating a Table with Spatial Data

    sqlCopyEditCREATE TABLE Locations (
        LocationID INT PRIMARY KEY,
        Name NVARCHAR(100),
        GeogLocation GEOGRAPHY
    );
   

   ---

   💡 Inserting Spatial Data (Latitude, Longitude)

    sqlCopyEditINSERT INTO Locations (LocationID, Name, GeogLocation)
    VALUES (1, 'New York', GEOGRAPHY::Point(40.7128, -74.0060, 4326));
   

   ✔️ Stores New York’s Latitude & Longitude.
   ✔️ 4326 → Represents the WGS 84 coordinate system (standard GPS).

   ---

   💡 Querying Spatial Data: Find Distance Between Two Locations

    sqlCopyEditDECLARE @Loc1 GEOGRAPHY = GEOGRAPHY::Point(40.7128, -74.0060, 4326);
    DECLARE @Loc2 GEOGRAPHY = GEOGRAPHY::Point(34.0522, -118.2437, 4326);
   
    SELECT @Loc1.STDistance(@Loc2) / 1000 AS DistanceInKm;
   

   ✔️ Calculates the distance between New York and Los Angeles (in KM).

   ---

   💡 Finding Nearby Locations (Within 10KM)

    sqlCopyEditSELECT Name, GeogLocation.STDistance(GEOGRAPHY::Point(40.7128, -74.0060, 4326)) AS Distance
    FROM Locations
    WHERE GeogLocation.STDistance(GEOGRAPHY::Point(40.7128, -74.0060, 4326)) < 10000;
   

   ✔️ Finds locations within a 10KM radius from New York.

   ---

   7.2 JSON and FOR JSON Query Clauses

   🔹 SQL Server natively supports JSON for storing and retrieving structured data.

   ✔️ Easier to integrate with APIs
   ✔️ Faster serialization and retrieval
   ✔️ Use FOR JSON to convert query results to JSON

   ---

   💡 Convert Table Data to JSON

    sqlCopyEditSELECT EmployeeID, Name, Salary
    FROM Employees
    FOR JSON AUTO;
   

   👉 Output:

    jsonCopyEdit[{"EmployeeID":101,"Name":"John","Salary":7500},
     {"EmployeeID":102,"Name":"Jane","Salary":8000}]
   

   ✔️ Automatically formats query results as JSON.

   ---

   💡 Manually Formatting JSON (FOR JSON PATH)

    sqlCopyEditSELECT EmployeeID, Name, Salary
    FROM Employees
    FOR JSON PATH, ROOT('EmployeeList');
   

   👉 Output:

    jsonCopyEdit{
      "EmployeeList": [
        {"EmployeeID":101,"Name":"John","Salary":7500},
        {"EmployeeID":102,"Name":"Jane","Salary":8000}
      ]
    }
   

   ✔️ FOR JSON PATH provides manual control over JSON structure.

   ---

   💡 Storing JSON in a Table

    sqlCopyEditCREATE TABLE Orders (
        OrderID INT PRIMARY KEY,
        OrderDetails NVARCHAR(MAX) CHECK (ISJSON(OrderDetails) = 1)
    );
   

   ✔️ Ensures only valid JSON is stored.

   ---

   💡 Querying JSON Data (JSON_VALUE & OPENJSON)

    sqlCopyEditDECLARE @json NVARCHAR(MAX) = '{"EmployeeID":101,"Name":"John","Salary":7500}';
   
    SELECT JSON_VALUE(@json, '$.Name') AS EmployeeName;
   

   👉 Output: John

   ✔️ Extracts values from JSON using JSON path notation.

   ---

   7.3 XML Data Types and FOR XML Query Clauses

   🔹 SQL Server supports XML data type for storing semi-structured data.

   ✔️ Use FOR XML to generate XML from queries
   ✔️ Use XML data type to store & query XML
   ✔️ Supports indexing for XML performance

   ---

   💡 Convert Table Data to XML

    sqlCopyEditSELECT EmployeeID, Name, Salary
    FROM Employees
    FOR XML AUTO;
   

   👉 Output:

    xmlCopyEdit<Employees EmployeeID="101" Name="John" Salary="7500" />
    <Employees EmployeeID="102" Name="Jane" Salary="8000" />
   

   ✔️ Formats query results as XML.

   ---

   💡 Custom XML Formatting (FOR XML PATH)

    sqlCopyEditSELECT EmployeeID, Name, Salary
    FROM Employees
    FOR XML PATH('Employee'), ROOT('EmployeeList');
   

   👉 Output:

    xmlCopyEdit<EmployeeList>
      <Employee>
        <EmployeeID>101</EmployeeID>
        <Name>John</Name>
        <Salary>7500</Salary>
      </Employee>
      <Employee>
        <EmployeeID>102</EmployeeID>
        <Name>Jane</Name>
        <Salary>8000</Salary>
      </Employee>
    </EmployeeList>
   

   ✔️ FOR XML PATH provides full control over XML formatting.

   ---

   💡 Storing XML in a Table

    sqlCopyEditCREATE TABLE Products (
        ProductID INT PRIMARY KEY,
        ProductDetails XML
    );
   

   ✔️ Allows storing structured XML data inside a column.

   ---

   💡 Querying XML Data

    sqlCopyEditDECLARE @xml XML = '<Product><ID>101</ID><Name>Laptop</Name></Product>';
   
    SELECT @xml.value('(/Product/Name)[1]', 'NVARCHAR(50)') AS ProductName;
   

   👉 Output: Laptop

   ✔️ Extracts values from XML using XPath notation.

   ---

   7.4 User-Defined Data Types

   🔹 User-Defined Data Types (UDTs) allow custom column definitions for consistency.

   ✔️ Ensures data type consistency across tables
   ✔️ Improves maintainability

   ---

   💡 Creating a User-Defined Data Type

    sqlCopyEditCREATE TYPE PhoneNumber FROM NVARCHAR(15) NOT NULL;
   

   ✔️ Now PhoneNumber can be used in table definitions.

   ---

   💡 Using UDT in a Table

    sqlCopyEditCREATE TABLE Customers (
        CustomerID INT PRIMARY KEY,
        ContactNumber PhoneNumber
    );
   

   ✔️ Ensures consistent phone number formatting across tables.

   ---

   💡 Dropping a User-Defined Type

    sqlCopyEditDROP TYPE PhoneNumber;
   

   ✔️ Removes the custom data type (if no columns are using it).

   ---

   🔎 Final Thoughts

   | Feature | Use Case | | --- | --- | | Spatial Data Types | Store & query geographical data (latitude, longitude) | | JSON Support | Integrate with APIs & store semi-structured data | | XML Data Types | Store structured documents inside SQL Server | | User-Defined Data Types | Enforce data consistency |

   ---

   Final Thoughts

   ✅ Use GEOGRAPHY for geospatial queries (distance, area calculations).
   ✅ Use JSON for flexible data storage & API integration.
   ✅ Use XML for structured, hierarchical data.
   ✅ Use User-Defined Data Types for enforcing consistency.

   🔎 Want More SQL Performance Tips?
   👉 Follow me on Hashnode for real-world SQL optimizations & best practices! 🚀

   ---

Data Integration and Automation

Integration with SQL Server Integration Services (SSIS)

1. 🔹 SQL Server Integration Services (SSIS) is an ETL tool for extracting, transforming, and loading data between databases, files, and external systems.

   ✔️ Automates data migration
   ✔️ Cleans and transforms data
   ✔️ Supports file imports, API calls, and cloud integration

   ---

   💡 Steps to Create an SSIS Package

   1️⃣ Open SQL Server Data Tools (SSDT)
   2️⃣ Create a New SSIS Project
   3️⃣ Drag Data Flow Task into Control Flow
   4️⃣ Add Source (SQL Server, CSV, XML, API, etc.)
   5️⃣ Add Destination (SQL Table, Azure, Excel, etc.)
   6️⃣ Run the SSIS Package

   👉 Use Case: Import CSV into SQL Table

   • Source: Flat File (CSV)

   • Destination: SQL Server Table

   • Tasks: Data Conversion, Lookup, Derived Columns

---

💡 Running SSIS Package from SQL Server

        sqlCopyEditEXEC SSISDB.catalog.start_package 
            @folder_name = 'MySSISProjects', 
            @project_name = 'SalesDataImport', 
            @package_name = 'ImportSalesData.dtsx';

✔️ Automates SSIS package execution within SQL Server.

---

8.2 OPENROWSET, OPENDATASOURCE, and Linked Servers

🔹 OPENROWSET & OPENDATASOURCE allow querying external data sources without setting up a Linked Server.

✔️ Access Excel, CSV, or Remote Databases
✔️ Ideal for one-time or ad-hoc queries

---

💡 Enabling Ad-Hoc Queries

        sqlCopyEditEXEC sp_configure 'show advanced options', 1;
        RECONFIGURE;
        EXEC sp_configure 'Ad Hoc Distributed Queries', 1;
        RECONFIGURE;

✔️ Allows OPENROWSET and OPENDATASOURCE functions.

---

💡 Querying an Excel File with OPENROWSET

        sqlCopyEditSELECT * FROM OPENROWSET(
            'Microsoft.ACE.OLEDB.12.0', 
            'Excel 12.0;Database=C:\Data\Sales.xlsx', 
            'SELECT * FROM [Sheet1$]'
        );

✔️ Reads data from an Excel file into SQL Server.

---

💡 Querying a Remote SQL Server with OPENDATASOURCE

        sqlCopyEditSELECT * FROM OPENDATASOURCE(
            'SQLNCLI', 
            'Server=RemoteServer;Database=SalesDB;Trusted_Connection=yes;'
        ).dbo.Customers;

✔️ Accesses data from a remote SQL Server without setting up a linked server.

---

🔹 Linked Servers: Permanent Remote Connection

🔹 Linked Servers allow querying remote databases as if they were local.

---

💡 Creating a Linked Server

        sqlCopyEditEXEC sp_addlinkedserver 
            @server = 'RemoteServer', 
            @srvproduct = '',
            @provider = 'SQLNCLI',
            @datasrc = 'RemoteServerName';

✔️ Establishes a permanent connection to a remote SQL Server.

---

💡 Querying a Linked Server

        sqlCopyEditSELECT * FROM RemoteServer.SalesDB.dbo.Orders;

✔️ Fetches data from a remote server without OPENROWSET.

---

8.3 BULK INSERT and BCP Utility

🔹 Bulk Loading Techniques help import millions of records efficiently.

✔️ Faster than INSERT for large datasets
✔️ Supports CSV, TXT, and BCP format files
✔️ Minimizes transaction logging for performance

---

💡 Using BULK INSERT for Fast CSV Import

        sqlCopyEditBULK INSERT Employees
        FROM 'C:\Data\Employees.csv'
        WITH (
            FORMAT = 'CSV',
            FIRSTROW = 2,
            FIELDTERMINATOR = ',',
            ROWTERMINATOR = '\n'
        );

✔️ Loads a CSV file into SQL Server.

---

💡 Exporting Data Using bcp (Command Line)

        shCopyEditbcp MyDatabase.dbo.Employees out C:\Data\Employees.txt -c -T -S MyServer

✔️ Exports SQL Server data into a text file.

---

💡 Importing Data Using bcp

        shCopyEditbcp MyDatabase.dbo.Employees in C:\Data\Employees.txt -c -T -S MyServer

✔️ Loads data from a text file into SQL Server.

---

8.4 Working with FileTables

🔹 FileTables allow storing & managing files inside SQL Server.

✔️ Stores PDFs, Images, Word Docs as database files
✔️ Accessible via Windows File Explorer
✔️ Supports Full-Text Search on stored files

---

💡 Enabling FileTables

        sqlCopyEditEXEC sp_configure 'filestream access level', 2;
        RECONFIGURE;

✔️ Enables FileStream for storing files in SQL Server.

---

💡 Creating a FileTable

        sqlCopyEditCREATE DATABASE FileDB
        CONTAINMENT = NONE
        ON PRIMARY (NAME = FileData, FILENAME = 'C:\FileTable\FileData.mdf')
        LOG ON (NAME = FileLog, FILENAME = 'C:\FileTable\FileLog.ldf')
        FILESTREAM (NAME = FileStreamData, FILENAME = 'C:\FileTable\FileStreamData', MAXSIZE = UNLIMITED)
        WITH FILESTREAM (NON_TRANSACTED_ACCESS = FULL, DIRECTORY_NAME = 'FileTableData');
        GO

        USE FileDB;
        CREATE TABLE Documents AS FILETABLE;

✔️ Creates a table where files can be stored like regular data.

---

💡 Inserting Files into a FileTable

1️⃣ Drag and Drop Files into the FileTable directory
2️⃣ Query Stored Files:

        sqlCopyEditSELECT Name, file_stream FROM Documents;

✔️ Files can be accessed through Windows & SQL Server.

---

💡 Querying FileTable Metadata

        sqlCopyEditSELECT Name, path_locator, file_type
        FROM Documents;

✔️ Retrieves metadata about stored files.

---

🔎 Summary: Data Integration & Automation Techniques

---

Final Thoughts

✅ Use SSIS for advanced ETL and data transformation.
✅ Use OPENROWSET and OPENDATASOURCE for ad-hoc remote queries.
✅ Set up Linked Servers for permanent external database access.
✅ Use BULK INSERT or bcp for high-speed data imports.
✅ Use FileTables for storing and querying documents inside SQL Server.

🔎 Want More SQL Performance Tips?
👉 Follow me on Hashnode for real-world SQL integration techniques! 🚀

---

1. Advanced Analytical and Statistical Functions in SQL Server 🚀

   SQL Server provides advanced analytical and statistical functions to perform complex calculations on datasets. These functions enhance reporting, forecasting, and time-series analysis.

   ---

   9.1 Statistical Functions 📊

   SQL Server includes built-in statistical functions for computing measures like variance, standard deviation, median, and percentiles.

   ✔️ STDEV() – Standard Deviation
   ✔️ VAR() – Variance
   ✔️ PERCENTILE_CONT() – Continuous percentile
   ✔️ PERCENTILE_DISC() – Discrete percentile
   ✔️ MEDIAN() – Returns the median value

   ---

   💡 Example: Standard Deviation and Variance Calculation

    sqlCopyEditSELECT 
        STDEV(Salary) AS StdDevSalary,
        VAR(Salary) AS VarianceSalary
    FROM Employees;
   

   👉 Output:

   | StdDevSalary | VarianceSalary | | --- | --- | | 1200.45 | 1441080.25 |

   ✔️ Standard Deviation (STDEV) shows data spread.
   ✔️ Variance (VAR) measures how far values are from the mean.

   ---

   💡 Example: Finding Median Salary

    sqlCopyEditSELECT PERCENTILE_CONT(0.5) 
        WITHIN GROUP (ORDER BY Salary) AS MedianSalary
    FROM Employees;
   

   ✔️ PERCENTILE_CONT(0.5) computes the median (continuous distribution).

   ---

   💡 Example: Finding 90th Percentile

    sqlCopyEditSELECT PERCENTILE_CONT(0.9) 
        WITHIN GROUP (ORDER BY Salary) AS P90Salary
    FROM Employees;
   

   ✔️ Returns the salary at the 90th percentile (continuous).

   ---

   9.2 LEAD, LAG, FIRST_VALUE, LAST_VALUE

   🔹 Window functions analyze previous, next, first, and last rows.

   ✔️ LEAD() – Accesses the next row’s value
   ✔️ LAG() – Accesses the previous row’s value
   ✔️ FIRST_VALUE() – Returns the first row’s value in a partition
   ✔️ LAST_VALUE() – Returns the last row’s value in a partition

   ---

   💡 Example: LEAD() and LAG() – Comparing Previous and Next Salaries

    sqlCopyEditSELECT EmployeeID, Name, Salary,
        LAG(Salary) OVER (ORDER BY Salary) AS PreviousSalary,
        LEAD(Salary) OVER (ORDER BY Salary) AS NextSalary
    FROM Employees;
   

   👉 Output:

   | EmployeeID | Name | Salary | PreviousSalary | NextSalary | | --- | --- | --- | --- | --- | | 1 | John | 5000 | NULL | 6000 | | 2 | Jane | 6000 | 5000 | 7000 | | 3 | Mark | 7000 | 6000 | 8000 |

   ✔️ LEAD() gets the next row’s value.
   ✔️ LAG() gets the previous row’s value.

   ---

   💡 Example: FIRST_VALUE() and LAST_VALUE()

    sqlCopyEditSELECT EmployeeID, Name, Salary,
        FIRST_VALUE(Salary) OVER (PARTITION BY DepartmentID ORDER BY Salary) AS LowestSalary,
        LAST_VALUE(Salary) OVER (PARTITION BY DepartmentID ORDER BY Salary ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING) AS HighestSalary
    FROM Employees;
   

   ✔️ FIRST_VALUE() returns the first row’s value.
   ✔️ LAST_VALUE() returns the last row’s value.

   ---

   9.3 Pivoting and Unpivoting Data 🔄

   🔹 PIVOT converts rows to columns (summarization).
   🔹 UNPIVOT converts columns to rows (normalization).

   ---

   💡 Example: Pivoting Sales Data

    sqlCopyEditSELECT * FROM 
    (SELECT EmployeeID, Month, SalesAmount FROM Sales) AS SourceTable
    PIVOT (
        SUM(SalesAmount) 
        FOR Month IN ([Jan], [Feb], [Mar])
    ) AS PivotTable;
   

   👉 Output:

   | EmployeeID | Jan | Feb | Mar | | --- | --- | --- | --- | | 1 | 5000 | 6000 | 7000 | | 2 | 4500 | 5500 | 6200 |

   ✔️ Converts month-wise sales into columns.

   ---

   💡 Example: Unpivoting a Pivoted Table

    sqlCopyEditSELECT EmployeeID, Month, SalesAmount
    FROM Sales_Pivoted
    UNPIVOT (
        SalesAmount FOR Month IN ([Jan], [Feb], [Mar])
    ) AS UnpivotTable;
   

   ✔️ Converts column-based sales back to rows.

   ---

   9.4 Advanced Time Series Queries 📅

   🔹 Time-series functions analyze trends over time.

   ✔️ DATEADD() – Adds/subtracts date intervals
   ✔️ DATEDIFF() – Computes the difference between two dates
   ✔️ LAG() for time-based comparisons
   ✔️ Window Functions for moving averages

   ---

   💡 Example: Sales Growth Calculation

    sqlCopyEditSELECT OrderDate, SalesAmount,
        LAG(SalesAmount) OVER (ORDER BY OrderDate) AS PreviousSales,
        (SalesAmount - LAG(SalesAmount) OVER (ORDER BY OrderDate)) / LAG(SalesAmount) OVER (ORDER BY OrderDate) * 100 AS GrowthRate
    FROM Sales;
   

   👉 Output:

   | OrderDate | SalesAmount | PreviousSales | GrowthRate (%) | | --- | --- | --- | --- | | 2024-01-01 | 10000 | NULL | NULL | | 2024-02-01 | 12000 | 10000 | 20.00 | | 2024-03-01 | 13000 | 12000 | 8.33 |

   ✔️ LAG() is used to compute previous month’s sales.

   ---

   💡 Example: 3-Month Moving Average

    sqlCopyEditSELECT OrderDate, SalesAmount,
        AVG(SalesAmount) OVER (ORDER BY OrderDate ROWS BETWEEN 2 PRECEDING AND CURRENT ROW) AS MovingAvg
    FROM Sales;
   

   ✔️ Computes a 3-month rolling average.

   ---

   🔎 Summary: Advanced Analytical Functions

   | Function | Use Case | | --- | --- | | STDEV(), VAR() | Compute standard deviation & variance | | PERCENTILE_CONT() | Find percentiles & median | | LEAD(), LAG() | Compare previous & next row values | | FIRST_VALUE(), LAST_VALUE() | Get first & last row in partition | | PIVOT / UNPIVOT | Reshape data for reporting | | Moving Averages | Time-series trend analysis |

   ---

   Final Thoughts

   ✅ Use Statistical Functions for data analysis (percentiles, standard deviation).
   ✅ Use Window Functions (LEAD(), LAG()) for row-by-row comparisons.
   ✅ Use PIVOT() and UNPIVOT() for transforming data.
   ✅ Use Time-Series Functions for forecasting and trend analysis.

   🔎 Want More SQL Performance Tips?
   👉 Follow me on Hashnode for real-world SQL optimizations & best practices! 🚀

   ---

   1. SQL Server Best Practices and Tools 🚀

      SQL Server provides various tools and techniques to write maintainable SQL code, optimize performance, manage version control, and automate maintenance tasks.

      ---

      10.1 Best Practices for Writing Maintainable SQL Code ✍️

      🔹 Writing Clean, Readable SQL Code Improves Maintainability and Debugging.

      ✅ Use Consistent Naming Conventions

       sqlCopyEdit-- Bad
       SELECT * FROM emp_table;
      
       -- Good
       SELECT * FROM Employees;
      

      ✔️ Use PascalCase or snake_case for table and column names.

      ---

      ✅ Always Specify Column Names in INSERT Statements

       sqlCopyEdit-- Bad
       INSERT INTO Employees VALUES (1, 'John', 5000);
      
       -- Good
       INSERT INTO Employees (EmployeeID, Name, Salary) VALUES (1, 'John', 5000);
      

      ✔️ Prevents issues when table structure changes.

      ---

      ✅ Avoid Using SELECT *

       sqlCopyEdit-- Bad
       SELECT * FROM Employees;
      
       -- Good
       SELECT EmployeeID, Name, Salary FROM Employees;
      

      ✔️ Improves performance by selecting only necessary columns.

      ---

      ✅ Use CTEs and Window Functions for Readability

       sqlCopyEditWITH HighSalaryEmployees AS (
           SELECT EmployeeID, Name, Salary
           FROM Employees
           WHERE Salary > 7000
       )
       SELECT * FROM HighSalaryEmployees;
      

      ✔️ CTEs make queries modular and easier to debug.

      ---

      ✅ Use Comments for Clarity

       sqlCopyEdit-- Retrieve top 5 highest-paid employees
       SELECT TOP 5 EmployeeID, Name, Salary
       FROM Employees
       ORDER BY Salary DESC;
      

      ✔️ Adds clarity to complex queries.

      ---

      10.2 Version Control with T-SQL Scripts 🛠️

      🔹 SQL version control helps track database changes and rollback when needed.

      ✅ Use Git for Storing SQL Scripts

       shCopyEditgit init
       git add CreateTables.sql
       git commit -m "Initial commit - added table creation script"
      

      ✔️ Keeps a history of schema changes.

      ---

      ✅ Use SQL Server Management Studio (SSMS) Source Control Integration

      • Install SQL Source Control in SSMS.

      • Connect to Git, Azure DevOps, or SVN.

      • Track schema and script changes.

---

✅ Use DBAtools for Automated Database Versioning

            shCopyEditInstall-Module -Name dbatools
            Export-DbaScript -SqlInstance localhost -Path "C:\DBBackups\"

✔️ Automatically version control schema changes.

---

10.3 SQL Server Profiler and Extended Events 🔍

🔹 SQL Profiler and Extended Events help diagnose performance issues.

✅ Using SQL Server Profiler to Trace Slow Queries

            sqlCopyEditsp_trace_create @TraceID OUTPUT, 0, 'C:\TraceOutput.trc';
            sp_trace_setevent @TraceID, 12, 1, 1; -- Captures SQL Statements
            sp_trace_setstatus @TraceID, 1; -- Starts the Trace

✔️ Monitors slow queries and deadlocks.

---

✅ Using Extended Events for Lightweight Monitoring

            sqlCopyEditCREATE EVENT SESSION MonitorQueries ON SERVER
            ADD EVENT sqlserver.sql_statement_completed
            ADD TARGET package0.ring_buffer;
            ALTER EVENT SESSION MonitorQueries ON SERVER STATE = START;

✔️ Captures SQL queries without high overhead.

---

10.4 Performance Tuning Tools ⚡

🔹 Optimizing query performance ensures faster response times.

✅ Using Query Store to Identify Slow Queries

            sqlCopyEditSELECT TOP 5 query_id, total_elapsed_time
            FROM sys.query_store_runtime_stats
            ORDER BY total_elapsed_time DESC;

✔️ Identifies the most time-consuming queries.

---

✅ Using Database Engine Tuning Advisor (DTA)

            shCopyEditdta -S MyServer -E -D MyDatabase -if QueryWorkload.sql

✔️ Suggests indexes and optimizations for workloads.

---

10.5 Automating Maintenance Tasks with SQL Agent 🔄

🔹 SQL Server Agent automates scheduled tasks like backups, index rebuilds, and alerts.

✅ Scheduling a Nightly Database Backup

            sqlCopyEditUSE msdb;
            EXEC sp_add_job @job_name = 'NightlyBackup';
            EXEC sp_add_jobstep @job_name = 'NightlyBackup', 
                @step_name = 'BackupDatabase',
                @command = 'BACKUP DATABASE SalesDB TO DISK = ''C:\Backups\SalesDB.bak'' WITH FORMAT;';
            EXEC sp_add_schedule @schedule_name = 'DailyMidnight', 
                @freq_type = 4, -- Daily
                @active_start_time = 000000;
            EXEC sp_attach_schedule @job_name = 'NightlyBackup', @schedule_name = 'DailyMidnight';
            EXEC sp_add_jobserver @job_name = 'NightlyBackup';

✔️ Runs automatic database backups every midnight.

---

✅ Rebuilding Indexes Weekly for Performance

            sqlCopyEditEXEC sp_msforeachtable 'ALTER INDEX ALL ON ? REBUILD';

✔️ Improves index efficiency and query speed.

---

🔎 Summary: Best Practices and Tools

---

Final Thoughts

✅ Follow best practices to keep SQL code clean and maintainable.
✅ Use Git and SSMS Source Control for versioning SQL scripts.
✅ Monitor performance using SQL Profiler & Query Store.
✅ Automate backups and maintenance with SQL Server Agent.

🔎 Want More SQL Optimization Tips?
👉 Follow me on Hashnode for real-world SQL best practices! 🚀

---