It has been an year since I have been working in presales. I have been helping various startups with my presales team to submit their bid for projects with their prospective client. When their prospective clients asked for a website or web application for their company be it customer facing or for internal users of their organization, these startups submitted their bid only for website development. Their bid did not include the data warehousing, reporting and analytics modules that are helpful for their clients.
I started to assist them in preparing the presales documents with a holistic application framework including various components to give the client a big picture of how their IT system will grow over the years and why they need to have data warehouse, reporting and analytics modules for their websites.
The clients rarely said no to those modules and some client just move these modules to phase 2 or 3 of the project. They wanted the website to be be developed to go to market soon.
The development team did not have this holistic understanding of the project. I am writing this post so that they can have the following framework with listed components. These are minimum components and you can expand the framework as per the clients requirements.
Website and Web API
It is recommended to develop website and web API using MVC framework. In .NET I use ASP.NET MVC and in PHP I use CodeIgniter or Laravel.
User Roles and Permissions
This is one of the time-consuming activities for developers to implement. My suggestion is to use OWIN Katana. In addition it is advisable to have screens that will help the admin of this website manage users, roles and permissions using OWIN Katana library to accomplish this. This may be time consuming to implement in the first project but keep it tested and ready for use in new projects. Another feature to add is the integration of various social media logins. I usually prefer to have Facebook, Twitter and Google login enabled.
Data Access Layer
I highly recommend developers to use ORM framework for data access layer. Write minimal lines of SQL code. Do not embed the SQL codes in the application. In the .NET world, I prefer to use Entity Framework as data access layer if the database is SQL Server. If the database is MySQL or PostgreSQL I prefer to use DevExpress XPO. This is an awesome framework. If you foresee that the client may be using different RDBMS for the application I highly recommend DevExpress XPO.
In .NET world I use LINQ queries and don’t write much of SQL. CodeIgniter also has ORM libraries to use.
For NoSQL databases like MongoDB use the recommended libraries for the ORM layer.
Logging
I recommend using NLog in .NET. There are tons of libraries available for logging. I recommend the ones that can write to file or database. At enterprise level I recommend logging into Cloud Platform. My preference is to use Google Cloud Stackdriver to log all messages for both on-premise or on-cloud applications. The architecture team must reinforce the logging to cloud platform.
It is also recommended to use services from Splunk or Sumologic for moving the log files from on-premise server to cloud and run analysis and anomaly detection etc.
Caching
It is recommended to have at least server level caching using HTTP caching library present in ASP.NET and other frameworks. The recommended solution is to use a centralized caching that the applications can share between them. I recommend using Google Cloud Memorystore which is based on Redis caching framework. You can also deploy an on-premise caching server and use Redis from that server.
SMTP Module
While developers by default use SMTP servers in their company I recommend using SendGrid service. This is scalable and this is easy to integrate to your application.
Analytics
It is necessary to understand how users are using the application we developed. Google Analytics is one of the best frameworks that will help us track how users use our application and their interaction with the application. Get reports from Google Analytics console and understand the value you are adding to the enterprise through your application.
Sessions
The ASP.NET sessions must be using SQL Server db to store all session details in a database so that no refactoring is required when scaling the application horizontally.
Even CodeIgniter supports storing session details in MySQL. You can check out my previous post for this.
Request Optimization
It is mandatory to use bundling of CSS, JS to minize the number of requests sent to the server to get the required CSS and JS files. It is also recommended to minify the CSS and JS files and use a version number a suffix in the name of the file and deploy it.
Subdomain Hosting
It is better to deploy the CSS, JS and images to the different subdomains even though the folders may be residing inside the web application. This will help browsers execute requests in parallel and get resources in multiple threads.
Content Delivery Network
CDNs are not that expensive these days. Use CDN from a cloud provider like Google. I am a big fan of Google Cloud products hence recommend the CDN from Google Cloud. Keep an eye on the billing because the bots may be increasing your bill.
Monitoring
Use monitoring tools outside of the machine which hosts the website and database to monitor its availability. At the minimum use services like Assertible and Runscope to monitor website pages and web API every hour or more frequently. Setup a different user agent for calls from these monitoring tools to exclude such traffic from Analytics report.
Supporting Applications
It is recommended to use messaging queues like MSMQ or Rabbit MQ to communicate between the web application and supporting applications. The supporting applications must not access database directly. They should use web API to access this.
Operations – CI/CD
Continuous Integration and Continuous Deployment is a prerequisite for the team to be agile. The missing piece that most teams wonder about is the unit tests and automated tests. Without these it is impossible to achieve the continuous delivery. Developers must help QA team to develop automated tests using framework like Nightwatch JS and developers must write unit tests for their code.
Data warehouse, reporting and analytics
Developers must build data warehouse, reporting and analytics solutions as a part of the project. In general these modules have the following components. This is a stack from Microsoft listed below.
Data warehouse – SQL Server Data Warehouse
ETL Framework – SSIS – SQL Server Integration Services
Reporting Solution – SSRS – SQL Server Reporting Services
Analysis Solution – SSAS – SQL Server Analysis Services.
You can also use the option from Google Cloud platform.
Data warehouse – Google BigQuery
ETL Framework – Talend, Google Cloud Data Fusion
Reporting Solution – Google Data Studio, Tableau, Microsoft Power BI.
Analysis Solution – BigQuery ML and other ML libraries.
This is the conclusion of this post. I know it is exhaustive. This is now a minimum required scope for a small company to expand their capabilities and add new set of services for the clients. The main goal of the project is to get the client’s foot into the cloud and there by expand the applications and services running from the cloud platform.
My role is to help companies build capabilities in these areas and be market ready and process-driven. If you need help you can use the chat box in the bottom right corner to ask me questions.
Recent Comments