Wednesday, May 29, 2019

Simple C-Sharp Console Application to convert log files to different file formats(.csv or .json)

          This article covers creating a simple console application that converts log file to different file formats easily with some customization. Though this is a console application but this can be developed and used as class library application which can be referenced as .dll file in any other .net application to serve for converting complex log files to user understandable formats and patterns. We are using Facade design pattern for the application structure.

Below is the prerequisite for the application:

1. Visual Studio 2015 or higher
2. Basic concepts of OOPs and dependency injections

 Lets start creating the application:

Step 1: Create projects name as "LogConvertor" as console application type and "LogConvertor.Application" as class library in Visual Studio.

  1. Create a new project in VS and select console application type from project templates selection screen and give it a name "LogConvertor". This will interact as user front end application.
  2. Create a new project in VS and select class library type from project templates selection screen and give it a name "LogConvertor.Application". This will handle file read, parse and conversion tasks.
  3. Create Convertor.cs class file in LogConvertor.Application project which will have a public method called Convert() as shown in below image:
  4. Convert Method
    Convert() Method
  5. Create Facade.cs static class file in LogConvertor project which will have static methods called as GetReader(), GetParser() and GetWriter() which will return reader, parser and writer objects for log conversion. Please refer below image:
  6. Facade class
    Facade Class and Methods
  7. Now in the Main() method of Program.cs file of LogConvertor project, add reference to the LogConvertor.Application namespace in namespace section at top.
  8. Then create a Convertor class object and call Convert() method by passing required parameters in it. Please follow the below image:
  9. Program Class
    Program Class
  10. This concludes the work done for LogConvertor and LogConvertor.Application projects.
Step 2: Create projects name as "LogConvertor.Implementation" as class library in Visual Studio. This project will consist of all implementation logic of log converter.
  1. Create FileReader.cs class file in LogConvertor.Implementation project which will have Read() to read log file line by line and return log lines collection as shown in below image:
  2. File Reader
    File Reader Class
  3. Create Parser.cs class file in LogConvertor.Implementation project which will have Parse() method to parse log line collection obtained from FileReader's Read() method. 
  4. Also add some helper methods to extract the log line data according to your log file content format. Please refer below image for more details on parsing:
  5. Parser Class Part 1
    Parser Class Part 1

    Parser Class Part 2
    Parser Class Part 2

    Parser Class Part 3
    Parser Class Part 3
  6. Now after reading and parsing the log lines of log file, we need to write those to new specified format. 
  7. for this article, we are converting logs into .csv file format and JSON objects as shown in below image:
  8. CSV Writer Class
    CSVWriter Class
    JSON Writer Class
    JSON Writer Class
  9. You can create same classes and methods for other file formats. 
  10. This concludes the work done for LogConvertor.Implementation project.


Step 2: Create projects name as "LogConvertor.Infrastructure" as class library in Visual Studio. This project will consist of all infrastructure level interfaces and collection object classes for log converter application.
  1. Create LogLine.cs and LogItem.cs classes for base infrastructure to hold and flow of log data values through application. Please refer below images for more details:
  2. LogLine Class
    LogLine Class
    LogItem Class
    LogItem Class
  3. We can change the LogLine and LogItem properties and code according to our requirements for reading, parsing and writing the new log at our own convenience.
  4. Also create a string utility class which will have a method to enclose the string value inside double quotes as in image:
  5. String Utility Class
    StringUtility Class
  6. Now Create IReader.cs, IParser.cs and IWriter.cs interfaces having return types and parameter list as shown in below images.
  7. IReader Interface
    IReader Interface
    IParser Interface
    IParser Interface
    IWriter Interface
    IWriter Interface
  8. At last, your solution folder structure and referenced projects structure should look like below:
  9. Project Solution Structure
    Project Solution Structure

    LogConvertor Referenced Projects
    LogConvertor Referenced Projects

    LogConvertor.Application Referenced Projects
    LogConvertor.Application Referenced Projects

    LogConvertor.Implementation Referenced Projects
    LogConvertor.Implementation Referenced Projects

  10. Finally, try and build the solution and you are good to go.

Scope and Conclusion: 
           This console application can be used to demonstrate how the facade design pattern works for the education purpose. There is a future scope of developing this application as class library type to serve as utility module which can be used to convert log files to user friendly readable format.