Cash Talks - BS Walks RSS 2.0
# Saturday, 01 January 2011

I had an interesting problem where I needed to protect text within Quoted Strings from a series of Regex search and replace operations.  Trying to write even simple regular expressions replace process that would exclude text in quotes was too painful for my understanding of Regex.  I’m sure there are gurus who can throw that out but I’m not one of them. 

While my job was more extensive than simply upper case, what if you wanted to change all characters in a string to upper case except for ones within double quotes?  For example:

The “quick brown fox” jumped over the “lazy dog”
would become
THE “quick brown fox” JUMPED OVER THE “lazy dog”

There are number of quoted string regex expressions out there and I needed one that provided for escaped “ within the string that had a preceding  \.

The following regex replace would replace quoted strings with the word STUFF

[regex]::Replace($value,'"([^"\\]*(\\.[^"\\]*)*)"', 'STUFF')

Not much help but that’s where the regex MatchEvaluator comes in to play.  It’s a function that gets called for each match and the text it returns is what gets used.  For example, what if I wanted to upper case only characters within double quotes?  The following code would make quick work of it.

$value = 'The "quick brown fox" jumped over the "lazy dog"'

 

$QuotedTextMatchEvaluator = [System.Text.RegularExpressions.MatchEvaluator]{

       $args[0].ToString().ToUpper()

}

 

[regex]::Replace($value,'"([^"\\]*(\\.[^"\\]*)*)"',$QuotedTextMatchEvaluator)

This produces
The “QUICK BROWN FOX” jumped over the “LAZY DOG”

Simple enough but as I said it was the opposite of what I needed and I was doing more interesting work than generating uppercase.  My idea was to grab all of the quoted strings, save them in an array and then restore them when I was done.  The save-quotedstrings function replaces each quoted string with a unique token then restore-quotedstrings replaces the token with the saved string.  While context will dictate what is possible as a token, I chose to use an ASCII 1, a rarely used non-visible character followed by an incrementing number for each quoted string.  save-quotestrings returns the tokenized string and the array of saved quoted strings.  restore-quotedstrings takes a tokenized string and the saved string array and restores them.

 

function save-quotedstrings([string]$value)

{

       $SavedStrings = @()

       $QuotedTextMatchEvaluator = [System.Text.RegularExpressions.MatchEvaluator]{

              $SavedStrings += $args[0].ToString()

              [char]1 + ($SavedStrings.Count)

       }

       [regex]::Replace($value,'"([^"\\]*(\\.[^"\\]*)*)"',$QuotedTextMatchEvaluator),$SavedStrings

}

 

function restore-quotedstrings([string]$value,[array]$savedStrings)

{

       (1..$savedStrings.Count) | %{$value = $value.Replace([char]1 + "$_",$savedStrings[$_-1])}

       $value

}

 

 

$value = 'This "is \"a\" test" of a "quoted string" saver'

$value

$value,$savedStrings = save-quotedstrings $value

$value

$value = $value.ToUpper()

$value

$value = restore-quotedstrings $value $savedStrings

$value

The preceding code generates the following output.

This "is \"a\" test" of a "quoted string" saver
This _1 of a _2 saver
THIS _1 OF A _2 SAVER
THIS "is \"a\" test" OF A "quoted string" SAVER

Note:  the _ is show instead of the non-displayable ASCII 1.

Saturday, 01 January 2011 00:16:06 (GMT Standard Time, UTC+00:00)  #    Comments [55] -
Powershell
# Thursday, 09 December 2010

Well it's been almost 3 years without a blog entry!  I had actually forgotten all the passwords and had to start at the hosting, set my ftp password, get in at the file system, hack into the DasBlog config files before finally getting my login going.

This time I've got these things in my encrypted LockBox application.  Even that doesn't leave me feeling safe.  The App is a little clucky and people have talked about losing everything on an upgrade. This will be another 'todo'!

Thursday, 09 December 2010 01:36:34 (GMT Standard Time, UTC+00:00)  #    Comments [988] -
Personal
# Saturday, 22 March 2008

Buck Hodges' VSTS 2005 and 2008: Building Database Projects with Team Build

While this is specifically about Team Build, it applies to any kind of automated build process using MSBuild.  We're still using CCNet as our primary Build engine but Team Build is looking more appealing with 2008.

Saturday, 22 March 2008 01:15:26 (GMT Standard Time, UTC+00:00)  #    Comments [37] -
Visual Studio | MSBuild
# Monday, 10 March 2008

In my previous blog entry, Easily setting Tags on Media Files using PowerShell, I described how to set the media tags used by common MP3 players.

The reason I had this need stemed from a frustration from RIPing Audio Books into MP3s.  For example, let's say I'm compressing a 10 CD book.  Even with "good" audio naming from Internet Naming sources, you get 10 different albums with numberd tracks from 1 to whatever.  Sometimes the naming fixes this and sometimes it does a miserable job.  Copying and renaming the files doesn't do the job.  When you RIP the CD the encoder creates Meta Tags Used by MP3 players.

When your favorite tool is a hammer...

I didn't even look for a tool to solve this problem so if there is one... that's not really the point.  I was curious if I could solve it very easily using Powershell.  This is what I want:

  • Copy all files to a single directory
  • Name all files with some base name with a Zero padded count.
  • Provide for a flexible means for setting Tags on all Files.  For example, Author
  • Set the Album Tag so all files will appear as a single album
  • Set Meta Track number to coorespond the the Count and Title to the Filename

When I RIP the CDs it easy to get them all under a single directory and organize the directories Alphabetically so the CDs are in order.  The individually named where they also sort alphabetically.  As such the following line will return all MP3s under a directory in the order they should be played:

dir -Recurse -Filter *.mp3

As such, I need a CmdLet where I can pipe these.  This CmdLet will also take an Array parameter if that is your preference.

function Copy-NumberFile
 
( [
system.IO.FileInfo] $file=$(Throw "File is required"
)
 
, [string] $BaseName=$(Throw "BaseName is required"
)
  ,
$DestDir=$(Throw "DestDir is required"
)
  , [
int] $Count=$(Throw "Count is required"
)
  , [
int] $Digits=
1
  , [
switch]
$Verbose
 
, [switch]
$PassThru
 
)
{
 
$nbrStr = "{0:d$Digits}" -f
$Count
 
$NewName =
"$($BaseName)$($nbrStr)$($file.Extension)"
 
$NewFileName = [System.IO.Path]::Combine($DestDir.FullName,$NewName
)
 
if ($Verbose
)
  {
   
Write-Host
"Copy $($file.fullname) to $NewFileName"
 
}
  
Copy-Item $file.FullName -Destination $NewFileName -PassThru:
$PassThru
}

Function
Organize-AudioBookFiles 
 
( [
System.IO.DirectoryInfo]
$DestDir
 
, [hashtable]
$Tags
 
, [string]
$BaseName
 
, [int]$CountStart=
1
  ,
$Digits=-
1
  , [
array]$Files
)
{
 
begin
 
{
    [
array] $fileList =
$null
   
function AddFile( $FileToCheck
)
    {
     
if ($fileToCheck -ne $null
)
      {
        [
system.IO.FileInfo] $file =
$FileToCheck
       
if (!$file.Exists) { Throw "File does Not Exist"
}
       
$file
     
}
    }
    
    if (!$DestDir.Exists
)
    {
      $DestDir.Create
()
    }
    
if ((dir $DestDir.FullName | Measure-Object).Count -gt
0)
    {
     
Throw
"Destination Directory $($DestDir.FullName) is not empty"
    
}
   
foreach ($file in $files
)
    {
     
$fileList += AddFile($file

    }
  }
 
 
process
 
{
   
$fileList += AddFile($_
)
  }

  end
 
{
    $Count =
$CountStart
    
if ($digits -lt
0)
    {
      [
int]$Digits = [System.Math]::Truncate(([System.Math]::Log10($fileList.Count + $Count)))+
1
    }
   
foreach ($file in $fileList
)
    {
     
$file = Copy-NumberFile -file $file -DestDir $DestDir -BaseName $BaseName
`
       
-Digits $digits -Count ($Count) -Verbose
-PassThru
     
$filename = $file.
Name
     
$Tags.Track =
$Count
     
$Tags.Title = $filename.Substring(0,($filename.length-$file.extension.length
))
     
SetMedia-Tags -file $file -Tags
$Tags
     
$Count
++
   
}
  }
}

I have a helper function "Copy-NumberFile".  While I'm not sure if I'll ever need it in another operation, I can imagine it.  It copies a file to a Destination Directory, renames it with a BaseName followed by a Number and preserving the extension.  The number of Zero padded digits is specified in the $Digits parameter.  The $PassThru allows the caller to get the FileInfo for the created file.

An example usage might look like:

$Tags = @{Artists="Book Author";Album="Book Title";Comment="This is a comment";Genres=@("Fiction")}
$files = dir -Recurse -Filter *.mp3
Organize-AudioBookFiles -Files $files -DestDir "C:\temp\Book" -BaseName "BookName " -Tags $Tags

Monday, 10 March 2008 02:27:18 (GMT Standard Time, UTC+00:00)  #    Comments [42] -
Powershell

This weekend I found the need to set the Media Meta Tags in MP3 files I have RIPed.  I found this Huddled Masses article for using the taglib-sharp library.  I find Hashtables as a convenient machine for passing Pthese kinds of adhoc Name\Value pairs.

[void] [Reflection.Assembly]::LoadFile(".\taglib-sharp.dll")

function SetMedia-Tags
 
( [System.IO.FileInfo] $File=$(Throw "File is required")
  , [
hashtable] $Tags=$(Throw "Tags are required"
))
{
  $media = [TagLib.File]::Create($file.FullName
)
  foreach ($tag in $Tags.keys
)
  {
    $media.Tag.$tag = $Tags[$tag
]
  }
  $media.Save
()
}

Example Usage:

$Tags = @{Artists="Max Headgroom";Album="Smoke";Comment="This is a comment";Genres=@("Crazy")}
SetMedia-Tags -File ".\file1.mp3" -Tags $Tags

I used this code in a more specific project for organizing Audio Book MP3s I'll describe in a subsequent post.

Monday, 10 March 2008 01:12:26 (GMT Standard Time, UTC+00:00)  #    Comments [36] -
Powershell
# Friday, 07 March 2008

One of the interesting things about having a blog is seeing what people are searching for when they navigate to an entry.  On at least a dozen occasions I've seen searches where people were obviously trying to invoke NAnt from PowerShell.  Because I've talked about both NAnt and PowerShell without speaking to this subject, they have been disappointed in what they've found in my blogs.  Indeed, when I've performed my own searches, I've noticed there isn't a lot of useful info on doing this subject.  Especially at detecting an error in the NAnt script.

This isn't the most comprehensive script but it does the basics:

function Invoke-NAnt ($NantFile,$target,[hashtable]$Properties)
{
  $sb = New-Object
"System.Text.StringBuilder"
 
if ($properties -ne $null)
  {
    foreach ($key in $Properties.Keys)
    {
      [void] $sb.Append( '"' + "-D:$key=$($Properties.$key)" + '" ' )
    }
  }
  
  nant "-f:$NantFile" $target $sb.ToString()

 
if (-not $?)
  {
    Throw
"Nant Failed"
 
}
}

Example Usage:

Invoke-NAnt Test.build "TestTarget" @{Test1="Value1";Test2="Value2"}

Test.Build contains:

<project>
  <
target name="TestTarget"
>
    <
echo message="Test1=${Test1}"
/>
    <
echo message="Test2=${Test2}"
/>
    <
fail message="This is a failure"
/>
  </
target
>
</
project>

Output looks like:

NAnt 0.85 (Build 0.85.2478.0; release; 10/14/2006)
Copyright (C) 2001-2006 Gerry Shaw
http://nant.sourceforge.net

Buildfile: file:///C:/Projects/PSNant/Test.build
Target framework: Microsoft .NET Framework 2.0
Target(s) specified: TestTarget

TestTarget:

     [echo] Test1=Value1
     [echo] Test2=Value2

BUILD FAILED

C:\Projects\PSNant\Test.build(6,6):
This is a failure

Total time: 0 seconds.

Nant Failed
At C:\Projects\PSNant\InvokeNant.PS1:19 char:10
+     Throw  <<<< "Nant Failed"

This particular script is designed to fail with the 'fail task'.  This is because I was having trouble detecting errors.

  if (-not $?)
  {
    Throw
"Nant Failed"
 
}

The $? variable is set to $true if the previous instruction succeeded.  As of yet, I'm not getting the actual failure message.  If someone can help with that, I'd appreciate it.

Other usage comments:

The $properties parameter takes a hash table that gets marshaled into -D:<propname>=<propvalue> line arguments. I find the syntax of @{Test1="Value1";Test2="Value2"} to be more natural for PowerShell.

This function will return an array of text lines which are the output from NAnt.  If you don't want them to pipe out you'll need to assign it to a variable or [void] it.

$NantOutput = Invoke-NAnt Test.build "TestTarget" @{Test1="Value1";Test2="Value2"}

Finally:  Obviously this needs to be dressed out a little more for supporting some useful parameters to NAnt like Logger and Default Framework. 

Friday, 07 March 2008 23:26:48 (GMT Standard Time, UTC+00:00)  #    Comments [1] -
Powershell
# Sunday, 24 February 2008

Good writeup on installing Ubuntu 7.10 on Virtual PC.  However, it hung for me right after the first screen with two different ISO downloads.  I was trying this at home on my AMD Vista Laptop.  Some comments on the page makes me suspicious that I'll need use my Virtual Server at work.

Update:

With an RDP connection to an XP system at work I was able to get the Ubuntu to install.  I'll conclude, for now, there is some issue with VPC on Vista.

The mouse continued to be flaky but it could be the system within system within system connection I was using.  I'm on a Vista laptop with a touchpad, running an XP VPC with VPN RDP connection to an XP System running the Ubuntu VPC! Now why can't my mouse work?! :)

I guess on Monday I'll see how well the mouse works when I can get direct access to the VPC.  Next hurdle... get it on the network.  Then to see if I can Admin the thing through Powershell!

Sunday, 24 February 2008 01:33:58 (GMT Standard Time, UTC+00:00)  #    Comments [32] -
Virtual Linux
# Saturday, 23 February 2008

I'm in the initial stages of attempting to control Linux servers from Windows using PowerShell.

The obvious path is using SSH.  The question is how.  Unfortunately with all the buzz around PowerShell Remoting, what I'm finding is a lot of the reverse... accessing PowerShell through SSH.  The reciprocity of this may make the reverse true but I haven't found the confirmation link yet.

The long way around the block might be through Cygwin and Putty.  The link is a little old but it shows a way.  I'm hoping for a direct path from PowerShell so I can minimize installation dependencies.

Update:

NetCmdlets is the "almost" direct path.  It looks good but too pricy for the way I want use it.  Besides, it's "just" a wrapper for rsh.exe.  Looks like I'll be attempting my own wrapper.  Anyone else try to do this?

Saturday, 23 February 2008 05:11:47 (GMT Standard Time, UTC+00:00)  #    Comments [49] -
Powershell | Virtual Linux
# Monday, 18 February 2008

The other day I was given a hard drive and asked to analyze what was on it.  While it was easy to determine it had over 300,000 files and about 10 Gigs I didn’t want to navigate through the whole thing looking in the hundreds of folders.  I figured a good thing to know was the number of files and size by extension.

 

Powershell to the rescue.  After several iterations of various techniques I came up with a CmdLet allowing me to measure any kind of object.  You can specify which property to use as the Group and for measurement.  I’m not sure what else I’ll use it for, but I love making things generic.

 

function MeasureGroup-Object

  ( [string]$group=$(Throw "Group Name is Required")

  , [string]$property=$(Throw "Property Name is Required")

  , $items

  )

{

  begin

  {

    function processItem($item)

    {

      $key = $item.$group

     

      if ($Aggregate.$key -eq $null)

      {

        $Aggregate.$key = @{Count=0;Sum=0}

      }

      $Aggregate.$key.Count += 1

      $Aggregate.$key.Sum += $item.$property

    }

 

    # Hash table to collect stats

    $Aggregate = @{}

 

    if ($items -ne $null)

    {

      foreach ($item in $items)

      {

        processItem $item

      }

    }

  }

  process

  {

    if ($_ -ne $null)

    {

      processItem $_

    }

  }

  end

  {

    function AddProperty ($object,$name,$value)

    {

      $member = new-object management.automation.PSNoteProperty $name,$value

      $object.psobject.members.Add($member)

    }

 

    foreach ($key  in $Aggregate.Keys)

    {

      $obj = new-object management.automation.psobject

      AddProperty $obj $group $key

      AddProperty $obj Count $Aggregate.$key.Count

      AddProperty $obj Sum $Aggregate.$key.Sum

      $obj

    }

  }

}

 

 

So let’s put it to work.  First you have to collect the objects you want to measure.

 

$files = Get-ChildItem -Recurse | where {$_ -is [System.IO.FileInfo]}

 

This will recursively collect all the files under the current directory.  It helps to filter out the Directories.  Because they don’t have Extensions, they would end up inflating the stats for files without extensions.

 

Now to use the MeasureGroup-Object.

 

$stats = MeasureGroup-Object Extension Length $files

 

The $stats variable will contain an array with an entry for each extension type.  Each entry will have an Extension, Count and Sum value.

 

The following will display the results but not in any particular order.

 

$stats

 

This will be more interesting.

 

# Get Top 5 file types by count

$stats | Sort-Object Count -Descending | Select-Object -First 5

 

Extension                Count                     Sum

---------                -----                     ---

.cs                        842                 3018481

.hxs                       765               980371275

.cab                       756              1801724080

.sql                       538                 6734980

.dll                       367                79635976

 

These are also interesting views.

 

# Get Top 5 file types by total Length

$stats | Sort-Object Sum -Descending | Select-Object -First 5

 

# Display all Extensions

$stats | Sort-Object Extension

 

Some comments about the implementation.

 

The End processing loops through the $Aggregate results and builds PSObjects with Note values.  The PSObjects play nice with the Sort, Select and default display.

 

Related Post: PowerShell Directory Size

Monday, 18 February 2008 00:57:25 (GMT Standard Time, UTC+00:00)  #    Comments [712] -
Powershell
# Sunday, 17 February 2008

This is a useful template for starting a CmdLet that needs to work in a PipeLine or by passing parameters.

 

Processing is implemented in the processItem sub-function.  All the “–ne $null” conditions handle the differences between pipeline and parameter invocation.

 

function Test-PipelineOrParm ($parm)

{

  begin

  {

    function processItem($item)

    {

      # Implement processing in this function

      "processItem $item"

    }

   

    if ($parm -ne $null)

    {

      foreach ($item in $parm)

      {

        "begin $(processItem $item)"

      }

    }

  }

  process

  {

    if ($_ -ne $null)

    {

      "process $(processItem $_)"

    }

  }

  end

  {

    Write-Host "Done"

  }

}

 

These two examples demonstrate calling the template.

 

Write-Host "Test Pipeline processing"

Get-ChildItem | Test-PipelineOrParm

 

Write-Host "Test Parm processing"

Test-PipelineOrParm $(Get-ChildItem)

 

Sunday, 17 February 2008 23:26:38 (GMT Standard Time, UTC+00:00)  #    Comments [76] -
Powershell
Archive
<2011 January>
SunMonTueWedThuFriSat
2627282930311
2345678
9101112131415
16171819202122
23242526272829
303112345
About the author/Disclaimer

Disclaimer
The opinions expressed herein are my own personal opinions and do not represent my employer's view in any way.

© Copyright 2017
Cash Foley
Sign In
Statistics
Total Posts: 20
This Year: 0
This Month: 0
This Week: 0
Comments: 5994
Themes
Pick a theme:
All Content © 2017, Cash Foley
DasBlog theme 'Business' created by Christoph De Baene (delarou)