I was recently taking a training course and the lab guide said “create some files in c:stuff”.
Sure, I *could* use Windows Explorer or Command Prompt or even a PowerShell command to do this. I guess the easiest way would be to use Windows Explorer, select a file, copy it (Ctrl+C), then paste it multiple times (Ctrl+V [repeat until your fingers are tired]).
But hey, I’m a geek and I like to script stuff (yes, for fun). But I’m not stupid so I didn’t create a script from scratch. Stéphane van Gulick, a PowerShell MVP, has a nice script at http://powershelldistrict.com/create-files/
However, I like to tinker and can’t leave good enough alone so I forked it and added some additional functionality.
I give you… Create-RandomFiles.ps1 with the following changes
- Added a Progress indicator
- files are created with different sizes
- TotalSize defaults to MB
- added optional name prefix
- added execution statistics
- replaced fsutil.exe with New-Object byte[]
| 123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191 | Function Create-RandomFiles{<#.SYNOPSIS Generates a number of dumb files for a specific size. .DESCRIPTION Generates a defined number of files until reaching a maximum size. .PARAMETER TotalSize Specify the total size you would all the files combined should use on the harddrive. This parameter accepts the following size values (KB,MB,GB,TB). MB is assumed if no designation is specified. 200KB 5MB 3GB 1TB .PARAMETER NumberOfFiles Specify a number of files that need to be created. This can be used to generate a big number of small files in order to simulate User backup specefic behaviour. .PARAMETER FilesTypes This parameter is not mandatory, but the following choices are valid to generate files with the associated extensions: Multimedia : “.avi”,”.midi”,”.mov”,”.mp3″,”.mp4″,”.mpeg”,”.mpeg2″,”.mpeg3″,”.mpg”,”.ogg”,”.ram”,”.rm”,”.wma”,”.wmv” Image : “.gif”,”.jpg”,”.jpeg”,”.png”,”.tif”,”.tiff”,”.bmp”,”.dib”,”.wmf”,”.emf”,”.emz”,”.svg”,”.svgz”,”.dwg”,”.dxf”,”.crw”,”.cr2″,”.raw”,”.eps”,”.ico”,”.pcx” Office : “.pdf”,”.doc”,”.docx”,”.xls”,”.xlsx”,”.ppt”,”.pptx” “.rtf”,”.txt”,”.csv”,”.xml”,”.mht”,”.mhtml”,”.htm”,”.html”,”.xps”,”.dot”,”.dotx”,”.docm”,”.dotm”,”.odt”,”.wps”,”.xlt”,”.xltx”,”.xlsm”,”.xlsb”,”.xltm”,”.xla”,”.ods”,”.pot”,”.potx”,”.pptm”,”.potm”,”.pps”,”.ppsx”,”.ppsm”,”.odp”,”.pub”,”.mpp”,”.vsd”,”.vsdx”,”.vsdm”,”.vdx”,”.vssx”,”.vssm”,”.vsx”,”.vstx”,”.vst”,”.vstm”,”.vsw”,”.vdw” Junk : “.tmp”,”.temp”,”.lock” Archive : “.zip”,”.7z”,”.rar”,”.cab”,”.iso”,”.001″,”.ex_” Script : “.ps1″,”.vbs”,”.vbe”,”.cmd”,”.bat”,”.php”,”.hta”,”.ini”,”.inf”,”.reg”,”.asp”,”.sql”,”.vb”,”.js”,”.css”,”.kix”,”.au3″ If Filestypes parameter is not set, by default, the script will create all types of files. .PARAMETER Path Specify a path where the files should be generated. .PARAMETER NamePrefix Optional. Allows prepending text to the beginning of the generated file names so they can be easily found and sorted. .PARAMETER WhatIf Permits to launch this script in “draft” mode. This means it will only show the results without really making generating the files. .PARAMETER Verbose Allow to run the script in verbose mode for debbuging purposes. .EXAMPLE .Create-RandomFiles.ps1 -TotalSize 1GB -NumberOfFiles 123 -Path $env:Temp -FilesTypes ‘Office’ -NamePrefix ‘~’ Generate in the user’s temp folder 123 randomly named office files all beginning with “~” which total 1GB. .EXAMPLE .Create-RandomFiles.ps1 -TotalSize 50 -NumberOfFiles 42 -Path C:Usersadministratordocuments Generate in the adminstrator’s documents folder 42 randomly named files which total 50MB. .NOTES -Author: Stéphane van Gulick -Email : [email protected] -Version: 1.0 -History: -Creation V0.1 : SVG -First final draft V0.5 : SVG -Corrected minor bugs V0.6 : SVG -Functionalized the script V0.8 : SVG -Simplified code V1.0 : SVG ===== Change History ===== based on http://powershelldistrict.com/create-files/ -Author: Chad Simmons -2015/12/04: added Write-Progress, files are created with different sizes, TotalSize defaults to MB, added name prefix, added execution statistics, replaced fsutil.exe with New-Object byte[], added additional filetypes .LINK http://www.PowerShellDistrict.com http://blogs.CatapultSystems.com#>[cmdletbinding()]param( [Parameter(mandatory=$true)]$NumberOfFiles, [Parameter(mandatory=$true)]$path, [Parameter(mandatory=$true)]$TotalSize, [Parameter(mandatory=$false)][validateSet(“Multimedia”,”Image”,”Office”,”Junk”,”Archive”,”Script”,”all”,””)][String]$FilesType=$all, [Parameter(mandatory=$false)]$NamePrefix=””) begin{ $StartTime = (get-date) $TimeSpan = New-TimeSpan -Start $StartTime -end $(Get-Date) #New-TimeSpan -seconds $(($(Get-Date)-$StartTime).TotalSeconds) $Progress=@{Activity = “Create Random Files…”; Status=”Initializing…”} Write-verbose “Generating files” $AllCreatedFilles = @() function Create-FileName { [CmdletBinding(SupportsShouldProcess=$true)] param( [Parameter(mandatory=$false)][validateSet(“Multimedia”,”Image”,”Office”,”Junk”,”Archive”,”Script”,”all”,””)][String]$FilesType=$all, [Parameter(mandatory=$false)]$NamePrefix=”” ) begin { $AllExtensions = @() $MultimediaExtensions = “.avi”,”.midi”,”.mov”,”.mp3″,”.mp4″,”.mpeg”,”.mpeg2″,”.mpeg3″,”.mpg”,”.ogg”,”.ram”,”.rm”,”.wma”,”.wmv” $ImageExtensions = “.gif”,”.jpg”,”.jpeg”,”.png”,”.tif”,”.tiff”,”.bmp”,”.dib”,”.wmf”,”.emf”,”.emz”,”.svg”,”.svgz”,”.dwg”,”.dxf”,”.crw”,”.cr2″,”.raw”,”.eps”,”.ico”,”.pcx” $OfficeExtensions = “.pdf”,”.doc”,”.docx”,”.xls”,”.xlsx”,”.ppt”,”.pptx” $OfficeExtensions2 = “.rtf”,”.txt”,”.csv”,”.xml”,”.mht”,”.mhtml”,”.htm”,”.html”,”.xps”, ` “.dot”,”.dotx”,”.docm”,”.dotm”,”.odt”,”.wps”, ` “.xlt”,”.xltx”,”.xlsm”,”.xlsb”,”.xltm”,”.xla”,”.ods”,` “.pot”,”.potx”,”.pptm”,”.potm”,”.pps”,”.ppsx”,”.ppsm”,”.odp”, ` “.pub”,”.mpp”,”.vsd”,”.vsdx”,”.vsdm”,”.vdx”,”.vssx”,”.vssm”,”.vsx”,”.vstx”,”.vst”,”.vstm”,”.vsw”,”.vdw” $OfficeExtensions += $OfficeExtensions2 $JunkExtensions = “.tmp”,”.temp”,”.lock” $ArchiveExtensions = “.zip”,”.7z”,”.rar”,”.cab”,”.iso”,”.001″,”.ex_” $ScriptExtensions = “.ps1″,”.vbs”,”.vbe”,”.cmd”,”.bat”,”.php”,”.hta”,”.ini”,”.inf”,”.reg”,”.asp”,”.sql”,”.vb”,”.js”,”.css”,”.kix”,”.au3″ $AllExtensions = $MultimediaExtensions + $ImageExtensions + $OfficeExtensions + $JunkExtensions + $ArchiveExtensions + $ScriptExtensions $extension = $null } process{ Write-Verbose “Creating file Name” switch ($filesType) { “Multimedia” {$extension = $MultimediaExtensions | Get-Random} “Image” {$extension = $ImageExtensions | Get-Random} “Office” {$extension = $OfficeExtensions | Get-Random } “Junk” {$extension = $JunkExtensions | Get-Random} “Archive” {$extension = $ArchiveExtensions | Get-Random} “Script” {$extension = $ScriptExtensions | Get-Random} default {$extension = $AllExtensions | Get-Random } } Get-Verb | Select-Object verb | Get-Random -Count 2 | %{$Name+= $_.verb} $FullName = $NamePrefix + $name + $extension Write-Verbose “File name created : $FullName” Write-Progress @Progress -CurrentOperation “Created file Name : $FullName” } end { return $FullName } }}#—————-Process———————————————– process { If ($TotalSize -match ‘^d+$’) { [string]$TotalSize += “MB” } #if TotalSize isNumeric (did not contain a byte designation, assume MB $Progress.Status=”Creating $NumberOfFiles files totalling $TotalSize” Write-Progress @Progress Write-Verbose “Total Size is $TotalSize” $FileSize = $TotalSize / $NumberOfFiles $FileSize = [Math]::Round($FileSize, 0) Write-Verbose “Average file size of $FileSize” $FileSizeOffset = [Math]::Round($FileSize/$NumberOfFiles, 0) Write-Verbose “file size offset of $FileSizeOffset” $FileSize = $FileSizeOffset*$NumberOfFiles/2 Write-Verbose “Beginning file size of $FileSize” while ($FileNumber -lt $NumberOfFiles) { $FileNumber++ If ($FileNumber -eq $NumberOfFiles) { $FileSize = $TotalSize – $TotalFileSize Write-Verbose “Setting last file to size $FileSize” } $TotalFileSize = $TotalFileSize + $FileSize $FileName = Create-FileName -filesType $filesType Write-Verbose “Creating : $FileName of $FileSize” $Progress.Status=”Creating $NumberOfFiles files totalling $TotalSize. Run time $(New-TimeSpan -Start $StartTime -end $(Get-Date))” Write-Progress @Progress -CurrentOperation “Creating file $FileNumber of $NumberOfFiles : $FileName is $FileSize bytes.” -PercentComplete ($FileNumber/$NumberOfFiles*100) $FullPath = Join-Path $path -ChildPath $FileName# Write-Verbose “Generating file : $FullPath of $Filesize” try{ #fsutil.exe file createnew $FullPath $FileSize | Out-Null $buffer=New-Object byte[] $FileSize #http://blogs.technet.com/b/heyscriptingguy/archive/2010/06/09/hey-scripting-guy-how-can-i-use-windows-powershell-2-0-to-create-a-text-file-of-a-specific-size.aspx $fi=[io.file]::Create($FullPath) $fi.Write($buffer,0,$buffer.length) $fi.Close() } catch{ $_ } $FileCreated = “” $Properties = @{‘FullPath’=$FullPath;’Size’=$FileSize} $FileCreated = New-Object -TypeName psobject -Property $properties $AllCreatedFilles += $FileCreated Write-verbose “$($AllCreatedFilles) created $($FileCreated)” Write-Progress @Progress -CurrentOperation “Creating file $FileNumber of $NumberofFiles : $FileName is $FileSize bytes. Done.” -PercentComplete ($FileNumber/$NumberOfFiles*100) $FileSize = ([Math]::Round($FileSize, 0)) + $FileSizeOffset }}end{ Write-Output $AllCreatedFilles Write-Output “Start time: $StartTime” Write-Output “Execution time: $(New-TimeSpan -Start $StartTime -end $(Get-Date))” #http://blogs.technet.com/b/heyscriptingguy/archive/2013/03/15/use-powershell-and-conditional-formatting-to-format-time-spans.aspx}} |
Enjoy!
One of the challenges in Operations Manager is providing an effective way to visualize information which does not require a significant amount of overhead from the Operations Manager administrator to implement or maintain. A common example of this is a free disk space dashboard or report. There are a variety of methods currently available to visualize this type of data but each of them has their own positives and negatives. This blog post will cover a variety of topics including methods to visualize this type of data in Operations Manager and will investigate Power BI and a simple example of how it can be leveraged to provide this type of functionality.
The sections of this blog post include:
- Visualizing disk space information using Operations Manager, OMS and 3rd party solutions
- Why Power BI?
- Starting with Power BI
- Getting data into Power BI
- Developing a simple report or dashboard
- Breaking down how the sample was built
Visualizing disk space information using Operations Manager, OMS and 3rd party solutions:
There are a variety of methods to display free disk space for systems in Operations Manager. These include:
- Microsoft includes some excellent pre-built dashboards and views which provide visualization for disk space conditions. An example from the Disk Capacity view within the Microsoft Windows Server management pack is shown below.
- Microsoft has two reports which assist with showcasing KPI’s for your servers including processor, disk, network and memory: Performance by System and Performance by Utilization. These are available in the Windows Server Operating System Reports management pack through the Performance by System and Performance by Utilization reports. A small sample is shown below.
- Community provided reports: There is a well written community report which provides free disk space for both Windows and Unix systems. I blogged on this previously here.
- Operations Manager custom dashboards: You can also visualize free disk space directly in Operations Manager through the built-in dashboard functionality and create your own custom dashboard for free disk information.
- Building your own report: Building one of these in Operations Manager hasn’t been simple but it still works.
Microsoft OMS also provides a solid method to display free space for systems. I blogged on this at: [CSF]
There are a variety of 3rd party vendors which can display free disk space for systems in Operations Manager. These include: (ordered by when they arrived in the market)
Why Power BI?
Power BI is an interesting choice as it makes it easy for someone who is not a databases focused SME to gather data and visualize that data. Additionally, you can choose what data you want your users to have access to, and they can develop their own reports or dashboards using an intuitive visualization experience.
Starting with Power BI
Power BI is available as a cloud based user experience from https://powerbi.microsoft.com (sign-in in the top right corner). Additionally there is a Power BI desktop client which you can download. To gather data from my Operations Manager DW the first step I took was to install the Power BI desktop client. For simplicity I installed this on the server which contained my OperationsManagerDW database. Installation was straight forward so the next step was to connect the desktop client up to my OperationsManagerDW database (please note, you can also connect to the OperationsManager database but that is not a recommended practice).
Getting data into Power BI
Once the desktop client is installed, the next step is to go get some data. In this case we want to choose “SQL Server Database”.
Next we point to the Operations Manager Data Warehouse (use the fully qualified name of the database server).
Specify credentials which have access to the data warehouse.
Next we need to choose our tables. This is the part that can be confusing if you aren’t familiar with the OperationsManagerDW.
The goal for my first Power BI report or dashboard is to show disk space information which I know is stored as a performance counter in Operations Manager. To do this we want to add the ManagedEntity table (which translates object numbers to their names – critical for what we want to do). Additionally, I want the underlying performance information which is stored as Raw, Hourly and Daily information.
If you are unfamiliar with the table structure, the option to “Select Related Tables” may be helpful. In my example this provided a total of 14 tables which were related to what I was looking for.
Please note, choosing what to query out of a source like the OperationsManagerDW is where most of the actual work will occur based on what I have learned since starting with this blog post. For future iterations such as reports which only show the most recent free disk space I would only want to return the counters specific to free disk space and only the most recent value for those counters. For this example blog post however, returning all performance data provides an insight into the variety of information which is available in the OperationsManagerDW database (IE: Every performance counter that OpsMgr gathers, including those gathered by 3rd party management packs!)
Developing a simple report or dashboard:
For me the toughest part of learning how to use Power BI up to the level I have so far is the first screen. We have data now (whoho!) but how do we actually do something with it? You start out with a data set on the left, a blank middle screen and the visualization and fields information shown below (all screenshots in this section are from the web version of Power BI).
The first place to start is with the “slicer” which his shown selected below.
The slicer lets us go through the data that we have and pick out what we want to show. From the fields we add the Path from vManagedEntity which shows us all entities from our set of data.
The slicer with a Path field added is shown below.
We want to filter on the name, so we drag the name field onto the filters.
And then we use Advanced filtering. Since I did not select the specific data that I was looking for (I gathered all performance information from the OperationsManagerDW) I now want to filter this data down to just the disk related counters.
By choosing the advanced filter I can now specify any objects which contain C: or D:
This now filters the content shown in the Slicer.
From there I can add visualizations such as a line chart.
And then I can choose the fields to display in that line chart.
The results in the line chart now automatically filtered to the objects that I choose in the slicer. It’s pretty simple once you get used to it! The example below shows a single system chosen in the slicer.
Breaking down how the sample was built:
The sample which I built is shown below. It includes a slicer, a clustered column chart, a card, and a line chart on the bottom. As I select different objects the different views adapt.
The example below shows this same view with different data selected (just the C drives from three of my servers). The line chart shows the average disk space averaged over the three C drives which are selected.
My configurations are shown below.
First for the slicer – we add the FullName and filter it on fields which contain ;c: or ;d:.
The clustered column chart includes the AverageValue from vPerfHourly, the Fullname from vManagedEntity and the Path from vManagedEntity. The title and other configurations were set using the paintbrush object highlighted below.
The card was configured using the FullName from vManagedEntity and a custom title was configured.
The line chart was configured with the AverageValue and DateTime information from vPerfHourly.
For additional interesting reading on this topic I recommend looking at the following:
Summary: Power BI introduces some extremely flexible methods to visualize data from a variety of sources including databases such as the OperationsManager DW. I highly recommend checking out Power BI and seeing what it can do for you at https://powerbi.microsoft.com/. I owe a huge thank you to Dieter Gasser for his introduction and walk through for how to use Power BI!
Update: I received the following feedback that I had missed a step on this process which if it is not done you do not get the relationship between the slicer and the chart. Details are provided below:
- You have to click on the “Manage Relationships” button and create a relationship between the vManagedEntity table and the perf table you want to pull from. (such as Perf vPerfHourly). Once you establish the relationship then the chart and slicing works.
- If this isn’t done, then the chart that you see will show the aggregated data from all of your drives…even if nothing is selected.
