Friday, July 10, 2020

Powershell Walkthrough



Courtesy  to Wayne Sheffield for  posting  blogs " A MONTH OF POWERSHELL"  with drafting an inspiration line " what better way is there to learn a subject than to write about it?” .
Saying that i thought to shadow the same blog to learn powershell encouraging everyone to do the same. 
I'm drafting these topics in a self way to understand in the FAQ format. 
What is PowerShell ?
Powershell is an Microsoft's command/scripting tool to automate  the process and adminster the server. It's designed  as a .net object based scripting environment . Everything in powershell is an object even the output. 

How do you launch PowerShell ?
Powershell can be launched by Start Menu/Programs/Accessories/Windows powershell or Windows Powershell ISE.

What is the command to see all the process running in your machine ?
Get-Process

If its need to filter with sqlserver then the below command can be used.
Get-Process | Where-object Processname -Eq 'sqlservr'

The pipe sends the output (remember that output is an object) from the cmdlet running on  the left side to the cmdlet running on the right side .
The above cmdlet also introduces Comparison operator (-).


Command to get the list of available drives ?
Get-PSDrive 
This returns the list of all the physical drives,HKCU & HKLM registry hives, a drive to the windows environment variables and others.

Explain about Variables in PowerShell ?
The syntax of the variable is the name of the variable proceeded by a dollar sign($) . Values are assigned to variables with equal sign (=). When the variable is assigned and values are passed to it , an appropriate .net data type is created.
For eg . $Myvariable=123 , here myvariable is the variable name and 123 is the values with system.int32 data type.

String variables are assigned by using quotation marks. Powershell accepts both single or double quotation marks . However there is a difference in how they operate. Single quotation marks use the literal values inside the quotation marks as the string. Double quotation marks will perform string substitution of other variables in the string.
Eg. $i=123
$s='hello'
S2="$i $s"
$s2
executing this will produce the below result.  
123 hello

You can use the long concatenation build string.
$q="select top 100"
$q=$q + ", Firstname "
$q=$q +", Lastname"
$q=$q +" from MyTable;"

 Or it can be used with here-string
$q=@"
select Firstname,Lastname from Mytable;
"@
$q
where using the here-string ,the string starts with @ and ends with @ which must be with a line itself. What's really neat with using a here-string is that PowerShell respects that all the line breaks,quotation marks (single or double) or white space withing the string that maintains that in the variable.

Special variables :
Powershell has a few variables that are automatically created.

Get-help about_automatic_variables --> Variables that store PowerShell state information.
Get-Help about_preference_variables --> Variables that customise PowerShell behaviour.
Get-Help about_environment_variables--> working with windows environment variables within powershell.

Explain Arrays
Arrays are data structures designed to store a collection of items, which can be of same data type or of different data types. Items to arrays can be assigned in a few different manners.

$A= 5 4 3 2 1      #specific items

$B=6..10  #range,Integars only
$c=@("String1","String2","String3") #Specific Seperate String Items
$D=@()
[array] $E="a;e;i;o;u;y" -split ";" #Specific Seperate String Items

Explain Hash tables ?
A hash table is a simply a Name-value pair.  The following example shows creating and adding to the hash table.
$z=@{"Colarado", ="Denver"; "Virginia"= "RichMond";
#z.Add{"Alaska","Fairbanks"}

Explain about security feature in powershell ?
Powershell  by default is in restricted mode. If any scripts to be executed with .ps1 file must have the right execution policy set. 
Below are the different execution policies in powershell.

Restricted --> Doesn't load configuration files or run scripts. This is the default execution policy.
AllSigned--> Requires that all scripts and configuration files must be signed by a trusted publisher,including scripts that you write on the local computer.
RemoteSigned--> Requires that all scripts and configuration files downloaded from the internet be signed by a trusted publisher.
Unrestricted--> Loads as scripts and configuration files. If you run an unsigned script from an internet then you are prompted for permission to run.
ByPass--> Nothing is blocked and there are no warnings or promps.
Undefined--> Removes the current assigned exeuction policy from the current scope. This parameter will not remove an exeuction policy that is set in a group policy settings.

To set a execution policy.
Set-ExecutionPolicy RemoteSigned


param(
  [string]$MyVariable
)


Command line Arguments
You can provide your script with command-line arguments by including a PARAM script block as the very first executable code in your script. A param script block would look like:

param(
  [string]$MyVariable
)

Script Block :
A script block represents  a precompiled block of script that can be used as a single unit. In powershell, the boundaries of script block is designated with curly -Brace characters. They can be nested and used anywhere.

Functions :
Functions are a pre defined script block that is assigned a name. When u call a assigned name, all of the script blocks are executed. A function includes the 'function' keyword,an optional scope,a name(that you select),optional parameters,and a script block that consists of one or more powershell commands. An example of a function is the following, which returns the username.
Function Get-CurrentUser
{
[System.Security.Principal.WindowsIdentity]::Get-Current().Name
}
Get-CurrentUser
Function names should follow the naming  rules that powershell uses especially the verb-noun syntax.
There are 4 types of parameters : named,positional,switch and dynamic. Parameters can be read from the command line or from the pipeline. Get-Help About-functions can used to see more information.


Modules :
Modules are a group of related functions. You create the module by 
1.Placing the related functions  into one script file.
2.Save the script file with .psm1 extension.
3. Move the file into the $ENV:PSModulePath directory.
4. Load the module with Import_module cmdlet.
5. Unload the module with Remove-Module cmdlet.
6. You can list the functions in the module with the Export-ModuleMember cmdlet.

Error Handling .
There are two methods of error handling 1) Trap function 2 ) Try-Catch-Function.
Try is a script block that is attempted to run.Catch is a 0-N script blocks to run if there is a error in the try script block. Finally is 0-1 script blocks to run after the completion of either the try or catch blocks above it. 
$handling any error that Occurs.
Trap 
{
#Hanndling the error
$err=$_.Exception
write-host ,@err.Message
While ($err.Innerexception)
{$err=$err.Innerexception
{
Write output $err.message
}#end of the scipt.
Break



Wednesday, April 29, 2020

How to capture deadlocks using SQL Server Profiler:

How to capture deadlocks using SQL Server Profiler:
To capture a deadlock, first connect to a SQL Server database. To open the SQL Profiler in SQL Server Management Studio:
  • Click on Tools
  • Click on SQL Server Profiler
  • Connect to the server on which we need to perform profiling
  • On the Trace Properties window, under General tab, select the blank template
  • On the Events Selection tab, select Deadlock graph under Locks leaf
Let’s execute the below queries:
In the following order:
  1. Execute the query in the blue rectangle
  2. Execute the query in green rectangle
  3. Execute the query highlighted in grey
  4. Execute the query highlighted in blue
Notice that the process ids are shown on the information bar in SQL Server:
The dead lock will be generated on execution of the 4th step:
For later analysis, this can be saved from File -> Export -> Extract SQL Server Events -> Extract Deadlock Events…
Analyzing the deadlock graph:
  1. Oval with a blue cross represents the transaction/process that was chosen as the deadlock victim by SQL Server
  2. The ovals represents the processes, the one without cross represents the transaction/process which completed successfully
  3. The deadlock priority is set to default i.e. 0
  4. We also have log used, this represents the transactional log used. If the transaction has done a lot of updates, the log size will be larger. Hence to roll the a transaction which has done a large number of updates would take a lot of cost. In our case, the deadlock victim is the one with less transaction log, because that would take less cost. The rectangles represents the resource nodes.
  5. The rectangles represents the resource nodes.
  6. The HoBtID (heap or binary tree id) associated with the resource node is used to find the database object involved in the deadlock from sys.partitions view by using the following query:
SELECT object_name([object_id]) from sys.partitions 
WHERE hobt_id = xxxxxx
  1. The arrows represent the types of locks we have on each resource node
  2. The notations X and S on the arrows represents the exclusive and shared locks

Wednesday, April 4, 2018

Troubleshooting Timeout errors in Reporting Services

Working with reports in Reporting Services, you can face timeout errors. This situation might happen when using very heavy reports. I will show bellow a list of different timeouts that could be affecting reports execution in Reporting Services and how you can adjust their values.

1.       If the timeout is produced by the query execution of the report, you can modify this value in the Report Designer. To find this setting:

·         Open the report using the report designer (Business Intelligence)
·         Select the Data tab
·         Select the dataset properties
·         Increase the value of the timeout box (or put the number in minutes for the query duration if empty)


                      image

2.       If the timeout is caused by the length of the execution of the report, you can change the Report Execution Timeout property by selecting the 'Do not timeout report execution' option

More info in:

3.       If the timeout is caused by the session, you can change the setting of the sessionState in the web.config file in the folder “C:\Programme\Microsoft SQL Server\MSSQL.3\Reporting Services\ReportManager”

 <sessionState mode="InProc" cookieless="false" timeout="180" />

Restart IIS (Start - Run…, type in iisreset and then click OK).

ON THE SERVER

4.       If the timeout is caused by the connection, you can change web site Connection Timeout to 7200 sec (default is 120 sec)

·         Open IIS Manager.
·         Right click the web site where Report Server is running.
·         Select Properties.
·         What is the setting for Connection timeout?
·         If it is less than 7200 you could change to the value 7200.

5.       In RSReportServer.config you could set the following values (theses are sample values, you can adjust your own):

<Add Key="ProcessRecycleOptions" Value="1"/> <!--Disabled-->
<Add Key="CleanupCycleMinutes" Value="36000"/> <!--10 Hours-->
<Add Key="SQLCommandTimeoutSeconds" Value="0"/> <!--None-->
<Add Key="MaxActiveReqForOneUser" Value="100"/>
<Add Key="DatabaseQueryTimeout" Value="0"/> <!--None-->
<Add Key="RunningRequestsScavengerCycle" Value="36000"/> <!--10 Hours-->
<Add Key="RunningRequestsDbCycle" Value="36000"/> <!--10 Hours-->
<Add Key="RunningRequestsAge" Value="30"/>

6.       Check the ExecutionTimeout setting within the web.config

\Program Files\Microsoft SQL Server\MSSQL.X\Reporting Services\ReportManager\
and
\Program Files\Microsoft SQL Server\MSSQL.X\Reporting Services\ReportServer)

Default is 9000 seconds, which is 2.5 hours.
In both web.config files check to see what the values are
(e.g., <httpRuntime executionTimeout = "9000" /> ).
You can alter this to 36000.

Maria Esteban

Reporting Services Support Engineer

Wednesday, May 17, 2017

How to Apply Patch in AlwaysOn Availability Group Configuration?

I am going to pick step by step for an Availability Group with one secondary replica.
  1. Make sure that we have taken good recent OS backup with system state (or VMware snapshot with SQL services stopped), a good recent backup of all databases and a successful completion of a checkdb on the primary node. {This is not mandatory, but to avoid “Ouch” moment}
  2. From the node acting as the primary replica (SQL1), change the failover mode to manual
  3. Refresh the affected databases on the secondary replica (SQL2) and make sure that everything is green on the dashboard.
  4. Apply the patch (service pack of CU) on SQL2.
  5. Repeat the Windows Update and/or software updates until all available patches are applied. Do not move on with the patching steps until all patches and post patch reboot and configuration tasks are completed.
  6. Double check that patches have been applied, the cluster is healthy and AlwaysOn Availability Groups are functional.
  7. Make sure that synchronization state is SYNCHRONIZED.
  8. Fail over the availability group to the secondary replica (SQL2).
  9. Refresh the affected databases on secondary Replica (former primary = SQL1) until the synchronization state is synchronized.
  10. Apply the patch (service pack of CU) on SQL1.
  11. Repeat the Windows Update and/or software updates until all available patches are applied. Do not move on with the patching steps until all patches and post patch reboot and configuration tasks are completed.
  12. Double check that patches have been applied, the cluster is healthy and AlwaysOn Availability Groups are functional.
  13. Make sure that synchronization state is SYNCHRONIZED.
  14. Fail over the availability group to the primary node (back to SQL1).
  15. Change the failover mode to Automatic now (which we changed in Step b)
In case things do not go as planned, you have followed step a) so you know what needs to be done.

Thursday, January 12, 2017

VLF in SQL SERVER

Thanks to original blogger Shay Attiya

How are VLFs created, truncated and deleted?

Today I chose to talk about Virtual Log Files or VLF in short. VLFs are the physical files that implement the log file. The log file’s log records are stored sequentially in a set of these physical files.
Creating a VLF:
Every time space is allocated for the log file (Initial creation or log growth) new VLFs are created behind the scenes. The number of new VLFs is determined by the amount of space allocated.
1) 0MB > Space Added >= 64MB ,  4 new VLFs
2) 64MB > Space Added >= 1GB ,  8 new VLFs
3) 1GB > Space Added,                 16 new VLFs
To demonstrate, I’ll use this script to create a DB with a log file in various sizes and auto growths. The database’s data file sizes is irrelevant.

To view the size and number of the VLFs created I’ll use sys.database_files and DBCC LOGINFO.

10MB:
Our log file is 10MB, and as expected, we have 4 VLF files. But if we sum the size of the files we’ll get to a total of 10,477,568 Bytes while 10MB are 10,485,760 Bytes. There are 8,192 Bytes (size of a page) missing. This space is used by the file header and it does not store log records. You can see it in the StartOffset for the first VLF. This is true for all log file sizes.
VLFs 10MB

Here are the results for bigger log files:
65MB:
VLFs 65MB

250MB:
VLFs 250MB

1025MB:
VLFs 1025MB

Ok, so now we know how VLFs are created, but before we continue, here’s a short overview on DBCC LOGINFO.
DBCC LOGINFO:
The fields that will be relevant to better understand the VLF behavior are:
FileId – If you have one log file, this number will be the same
FileSize – This number is in Bytes
StartOffset – This number is in Bytes
FSeqNo – The file sequence number. The VLF with the highest FSeqNo is the one where current log records are being written into. FSeqNo = 0 indicates that the file hasn’t been used yet.
Status – Two possible values: 0 – VLF is recyclable, 2 – VLF may be active (Highest FSeqNo must be active)
Parity – Two possible values: 64 and 128. Every time a VLF is reused, the parity value is switched.
CreateLSN – Indicates at what LNS (Log Sequence Number) the VLF was created. CreateLSN = 0 states that the VLF was created at the time the log was created. Files with the same CreateLSN were created at the same time.
To see how the VLFs are being written on, I’ll create some activity in the DB.
The DB is set for simple recovery mode for now.

The empty VLF files will be filled one by one until they are all full.
VLFs FullLog

Notice that the first 2 VLF statuses are 0. This means that these VLFs are recyclable and can be written on.
After generating more activity the first VLF has been overwritten and its FSeqNo changed from 22 to 26. Now the 3 other VLFs are recyclable and have status = 0.
VLFs FullLog2
Log Truncation:
Truncation is when inactive VLFs are marked as recyclable. This is a logical operation. Truncation occurs in the following events:
1) When a checkpoint occurs (only in auto truncate mode)
2) When the log is backed up (not in COPY_ONLY)
3) When the recovery mode is set to SIMPLE
Auto Truncate Mode:
In Auto Truncate VLFs are being overwritten once they become inactive. The log file is in this mode when the DB is in SIMPLE recovery mode, or in FULL but no backup has been made.
Adding new VLFs (Growing the Log):
In order to make the log file grow and create more VLFs we must first get out if the Auto Truncate mode. To do that, we’ll set the DB to FULL recovery mode and back it up.

Once we generate some more activity, all of the VLFs are full and new ones are created.
VLFs MoreVLFs
The log file grew by 6MB and 4 new VLFs are added to it. All of the files that have already been written on (FSeqNo > 0) are marked as active (Status = 2) and cannot be truncated.
Forcing Truncation:
In order to be able to recycle the VLFs, we’ll back up the Log File.
After it has been backed up, all of the VLFs are marked as inactive (Status = 0) except for the VLF that is being used at the moment. Note that the log file hasn’t shrunk, and is still 16MB.
VLFs Truncated1

Now new log records can be written on the truncated VLFs.
VLFs OverWrite
Deleting VLFs (Shrinking the Log):
If the log file is too big, it can be shrunk by removing inactive VLFs from it. To do that we’ll make the DB go into Auto Truncate mode by set the recovery mode back to SIMPLE. If we are already in SIMPLE recovery mode, we can force a truncation by using the CHECKPOINT command.
VLFs Truncated2

Now we can shrink the log file
VLFs Shrunk
The file has shrunk to 6MB instead of 5 like we wanted. The reason is not all of the inactive VLFs are being removed during the file shrinking. In addition, the size of a single VLF can never change.
Now What?
Now after we have a better understanding of how VLFs are created, truncated and deleted, and have also learned how to control these events, we as DBAs can modify our database for better performance by controlling the number of VFLs in the log file.
But what do we want to achieve?
First, let’s consider our options:
Many VLFs:
Having a lot of relatively small VLFs means that the log file has more flexibility during normal activity, but may cause problems during recovery and restore, and also during CHECKPOINT when in Auto Truncate mode. All of the inactive VLFs are being inspected at these events, and having a lot of then may be time consuming.
In SQL Server 2012 an error log message will appear when going over 1,000 VLFs.
Few VLFs:
Having only a few big VLFs could cause problems involving truncation. A small amount of log records can prevent the VLF from being truncated. Shrinking the log file might suffer the same problems.
Bottom Line:
There is no conclusive best Practice when it comes to determining the desired number of VLF files in the log. DBAs who have tested the subject report the “Sweet Spot” is around 100 VLFs. Like every other aspect of DB design, you should test it on your own system and configure it for what suits you best.
If you have tested it already, I’d like it if you could share your insights.