ビルド・インサート文powershell to SQL

Question

私はすべてのファイルとその属性を特定のディレクトリから再帰的に書き込むpowershellスクリプトを用意しています。これは動作しますが、ディレクトリには1,000,000個ものファイルが存在する可能性があります。私がやりたいことは、トランザクションごとに1000個の挿入をバッチすることです。ここではオリジナルのPSは以下のとおりです。

$server = "" 
$Database = "" 
$Path = "C:\Test" 

$Connection = New-Object System.Data.SQLClient.SQLConnection 
$Connection.ConnectionString = "server='$Server';database='$Database';trusted_connection=true;" 
$Connection.Open() 
$Command = New-Object System.Data.SQLClient.SQLCommand 
$Command.Connection = $Connection 

foreach($file in Get-ChildItem -Verbose -Recurse -Path $Path | Select-Object Name,Length,Mode, Directory,CreationTime, LastAccessTime, LastWriteTime) { 
    $fileName = $file.Name 
    $fileSize = ([int]$file.Length) 
    $fileMode = $file.Mode 
    $fileDirectory = $file.Directory 
    $fileCreationTime = [datetime]$file.CreationTime 
    $fileLastAccessTime = [datetime]$file.LastAccessTime 
    $fileLastWriteTime = [datetime]$file.LastWriteTime 

    $sql = " 
       begin 
        insert TestPowerShell 
        select '$fileName', '$fileSize', '$fileMode', '$fileDirectory', '$fileCreationTime', '$fileLastAccessTime', '$fileLastWriteTime' 
       end 
      " 
    $Command.CommandText = $sql 
    echo $sql 
    $Command.ExecuteNonQuery() 
} 

$Connection.Close() 

私の考えは、ループの外にジャンプし、実行し、それが1000に到達するまでのインサートを追加しておくとしますカウンターのいくつかの並べ替えを実装することです。私はこの現在のセットアップで、1000でバッチを実行する方法を知ることができず、get-childitemループを実行してから実行します。興味がある人々のために

Answer 1

- これはそれを行うための一つの方法である。このような

function WriteBatch { 
    echo $sql 
    $Command.CommandText = $sql 
     $Command.ExecuteNonQuery() 
} 

$server = "" 
$Database = "" 
$Path = "" 
$Counter = 0 

$Connection = New-Object System.Data.SQLClient.SQLConnection 
$Connection.ConnectionString = "server='$Server';database='$Database';trusted_connection=true;" 
$Connection.Open() 
$Command = New-Object System.Data.SQLClient.SQLCommand 
$Command.Connection = $Connection 
[string]$sql =  " 
       begin 
        insert into TestPowerShell(NameString, FileSize, Mode, Directory, CreationTime, LastAccessTime, LastWriteTime) 
        values " 

foreach($file in Get-ChildItem -Verbose -Recurse -Path $Path | Select-Object Name, Length, Mode, Directory, CreationTime, LastAccessTime, LastWriteTime) { 
    $fileName = $file.Name 
    $fileSize = ([int]$file.Length) 
    $fileMode = $file.Mode 
    $fileDirectory = $file.Directory 
    $fileCreationTime = [datetime]$file.CreationTime 
    $fileLastAccessTime = [datetime]$file.LastAccessTime 
    $fileLastWriteTime = [datetime]$file.LastWriteTime 
    $sql = $sql + "('$fileName', '$fileSize', '$fileMode', '$fileDirectory', '$fileCreationTime', '$fileLastAccessTime', '$fileLastWriteTime')," 
    $sql += "`n" 
    $Counter++ 

    If($Counter -eq 900) { 
     $sql = $sql.Trim().Trim(',') 
     $sql = $sql + " End" 
     WriteBatch 
     $Counter = 0 
     $sql = " 
       begin 
        insert into TestPowerShell(NameString, FileSize, Mode, Directory, CreationTime, LastAccessTime, LastWriteTime) 
        values " 
    } 

} 

if ($Counter -gt 0){ 
    $sql = $sql.Trim().Trim(',') 
    $sql = $sql + " End" 
    WriteBatch 
} 

$Connection.Close() 

Answer 2

何かが行う必要があります。

function Execute-SqlQuery($query){ 
    Write-Host "Executing query:" 
    Write-Host $query; 
} 

$data = @(1,2,3,4,5,6,7,8,9,10,11); 
$batchSize = 2; 
$counter = 0; 
$sql = ""; 

foreach($item in $data){ 

    if($counter -eq $batchSize){ 
     Execute-SqlQuery $sql; 
     $counter = 0; 
     $sql = ""; 
    } 

    $sql += "insert into myTable(id) values($item) `n"; 
    $counter += 1;  
} 

Execute-SqlQuery $sql; 

Answer 3

$server = "" 
$Database = "" 
$Path = "C:\Test" 

$Connection = New-Object System.Data.SQLClient.SQLConnection 
$Connection.ConnectionString = "server='$Server';database='$Database';trusted_connection=true;" 
$Connection.Open() 
$Command = New-Object System.Data.SQLClient.SQLCommand 
$Command.Connection = $Connection 

# new variables to handle batching 
$batchcounter=0 
$batchsize=1000 
$sqlValues = New-Object Collections.ArrayList 

foreach($file in Get-ChildItem -Verbose -Recurse -Path $Path | Select-Object Name,Length,Mode, Directory,CreationTime, LastAccessTime, LastWriteTime) { 
    $fileName = $file.Name 
    $fileSize = ([int]$file.Length) 
    $fileMode = $file.Mode 
    $fileDirectory = $file.Directory 
    $fileCreationTime = [datetime]$file.CreationTime 
    $fileLastAccessTime = [datetime]$file.LastAccessTime 
    $fileLastWriteTime = [datetime]$file.LastWriteTime 

    $sqlValues.Add("('$fileName', '$fileSize', '$fileMode', '$fileDirectory', '$fileCreationTime', '$fileLastAccessTime', '$fileLastWriteTime')") 

    $batchcounter++ 

    # if the counter hits batchsize, run the insert, using lots of: 
    # insert into table 
    # values (1,2,3) 
    #  , (4,5,6) 
    #  , (7,8,9) 

    if ($batchcounter % $batchsize -eq 0) { 
     $sql = "insert TestPowerShell values {0}" -f ($sqlValues.ToArray() -join "`r`n,") 
     $Command.CommandText = $sql 
     Write-Host $sql 
     $Command.ExecuteNonQuery() 
     $sqlValues.Clear() 
    } 
} 

# catch any remaining files 
if ($batchcounter -gt 0) { 
    $sql = "insert TestPowerShell values {0}" -f ($sqlValues.ToArray() -join "`r`n,") 
    $Command.CommandText = $sql 
    Write-Host $sql 
    $Command.ExecuteNonQuery() 
    $sqlValues.Clear() 
} 

$Connection.Close()