Skip to main content

Get the maximum length of every field in a CSV, Excel etc

While setting up tables in our data warehouse for use with some external APIs, it was getting a little tedious to figure out the max length of some long fields to get the write size for CREATE TABLE statements, so I wrote this little script to calculate it for me. Yeah, it's kinda messy, so sue me (or don't). You can swap out the import-csv part to also import from Excel or whatever else

$csv = import-csv ~\Downloads\whatever.csv
$fields = $csv | select -first 1 | get-member | where MemberType -eq 'NoteProperty'

foreach($field in $fields)
{
    $data = $csv | select -ExpandProperty $field.Name
    $lengths = $data | % {$_.Length}
    $max = ($lengths | measure -Maximum).Maximum
    $field | add-member -NotePropertyName MaxLength -NotePropertyValue $max
}

$fields | select Name, MaxLength | Format-Table -AutoSize

Comments

Popular posts from this blog

Using WiX to create an event source during install of a .NET framework project

Edit: so I guess I wasn't the only one confused with this stuff, as it's been my most popular post by far!  If I've helped you out or saved you some time, please let me know in the comments :)

In order for this to work, you have to add references to WixUtilExtension and WixNetFxExtension to your WiX project.  Once that's done, add this inside a <Component> element:

<Util:EventSourcexmlns:Util="http://schemas.microsoft.com/wix/UtilExtension"Name="EVENTSOURCEGOESHERE"Log="Application"EventMessageFile="[NETFRAMEWORK40FULLINSTALLROOTDIR]EventLogMessages.dll" />
Obviously replace EVENTSOURCEGOESHERE with your event source name.  NETFRAMEWORK40FULLINSTALLROOTDIR is a property set by the WixNetFxExtension which stores the path to the .NET framework v4 directory, but you can replace this with the corresponding property for the directory containing the relevant EventLogMessages.dll file.  So if you're using the .NET framewo…

Using Log4Net to use both event log and a rolling log file

Here's the config section, note that the applicationNameproperty in the EventLogAppender needs to be the same as the event source in the windows event log that you want to log to.  If the event source doesn't exist, that appender won't work.  In this particular project I create that during install using WiX (which is covered in another post)

  <log4netdebug="true">
    <appendername="RollingLogFileAppender"type="log4net.Appender.RollingFileAppender">      <filevalue="log.txt" />      <datePatternvalue="dd-MM-yyyy" />      <appendToFilevalue="true" />      <locationinfovalue="false" />      <rollingStylevalue="Size" />      <maximumFileSizevalue="1MB" />      <maxSizeRollBackupsvalue="10" />      <staticLogFileNamevalue="true" />      <layouttype="log4net.Layout.PatternLayout">        <conv…

"A section using 'configSource' may contain no other attributes or elements" error after installing Application Insights

After installing the Application Insights nuget package to an Umbraco solution, you'll get this error:

A section using 'configSource' may contain no other attributes or elements

<ExamineLuceneIndexSets configSource="config\ExamineIndex.config" />
     <log4net configSource="config\log4net.config">
         <root>
             <level value="ALL" />
             <appender-ref ref="aiAppender" />
Source File: \project\web.config

This happens because part of the Application Insights installation process adds a <log4net> section to web.config.  Which would make sense, except Umbraco already has a <log4net> section in /config/log4net.config.  So as you can imagine, the solution is to manually move everything its added into that file. Unfortunately you can't just copy/paste the whole lot, but it's not particularly complicated:


Move <appender-ref ref="aiAppender" /> into the lo…