info

android app development service in India

Five reasons for your company to invest in mobile apps

A lot is heard and talked about modernization, internet, connection, etc. And a big question for many entrepreneurs is whether it’s worth investing in mobile apps.

In this article, we will address exactly that, understanding the importance and relevance of technologies that improve our business concept and increase intuitive communication with our customers, something that is really worth knowing and understanding!

Mobile apps have become indispensable in people’s lives. For businesses, it’s no different: knowing the reasons for investing in mobile apps can have several benefits. In addition to expanding the options for the consumer, it also helps to strengthen the brand.

Despite this, many entrepreneurs still neglect the importance of such tools. So that you don’t fall behind the competition, we have an article covering just that. Five reasons for your company to invest in mobile apps especially android app development service in India – thus bringing something unique to your business.

Investing money in fixed or variable income without leaving home is possible with financial application apps. They let you track trades and yields right at your fingertips, and some even provide information to learn more about investing.

1. Greater involvement and knowledge of the profile and behavior of users

The practicality of the applications allows for greater proximity and involvement of customers with companies or even employees with companies (in this case and applications for internal use), since access is simpler and faster.

Furthermore, through it, it is easier to monitor user behavior and offer products, services and solutions that are more attractive to their profile.

2.  More agility and efficiency in what the App proposes

If the business is sales and retail, for example, the application already provides the freight quote, shows the update of the delivery status, speeds up the filling out of digital records and allows the exchange of messages with faster responses.

Unlike websites and digital platforms, applications, when well structured, are easier, more intuitive and simple to use. In this way, it is possible for a wide range of potential consumers to know your product/service.

With just one click, the person accesses your app. In addition, the fact that it is installed on the users’ mobile device allows them to have their company as a reference – simply accessing the application when they need their business.

3. Improvements in processes or steps 

One of the main benefits of the applications is to offer greater control over processes, with access to information and optimization in product collection and delivery operations. A good application also needs to facilitate the users’ routine, this should be the essential basis of its construction.

4.  Direct communication channel

A good app facilitates communication with the consumer, as it offers a connection and an experience superior to those of websites. In addition, it makes it possible to send notifications, for example, which helps to increase the app’s open rate by attracting the interest of customers.

5. Greater control and monitoring of performance with reports and dashboard

Another advantage of having an app is that it can automatically generate reports and dashboards for daily performance measurement. In addition, the system can also be created to facilitate some internal company process, such as inventory control, financial management, organization of demands, employee evaluation, distance learning, etc.

iOS app development service in India

Is Hybrid App Development right for your Business?

The decision between developing a hybrid app versus developing a Native app is significant for the life of the app and it is almost impossible to go back after making the decision. Each has advantages and disadvantages, so let’s get some order.

The most significant advantage is of course that in a hybrid application you can write code once and use it on both the iOS system and the Android system. Compared to the Native app where it is necessary to develop 2 separate apps, one for each operating system.

The decision between them depends on the infrastructure on which the application will be developed, on the basis of this decision the developers will be recruited with the appropriate skills. Every project in its infancy has a budget and a small margin of error therefore, a wrong choice in the way the app is developed can jeopardize the whole project. For this, we are here to explain and help in this important decision.

Hybrid app – let’s talk about it.

So as we have already said, a hybrid application is developed by an innovative platform that enables the development of a single core code by WEB languages ​​for example: Java Script, CSS3, HTML5. Development on a browser in programming languages ​​familiar to almost everyone is designed to “bypass” the adaptation to any operating system. That is, developers are required to develop only one application, one language, for all operating systems. How does this contribute to you? Saving time, saving resources, simple dealing with a single factor.

Is a hybrid app right for me?

There are many parameters that need to be considered in deciding what type of app to develop. This decision has a huge impact on the development time, costs and lifecycle of the app. To facilitate the process first, you need to formulate the purpose of the app, its target audience, business objectives and technological requirements. Then, ask yourself, what is your budget? What is the level of urgency of launching the app? Does the app require heavy and complicated processing?

If your app mainly provides content of various types and does not include particularly complicated graphics and animations, a hybrid app is the solution for you!

You have decided to develop your application in a hybrid way – what is the next step?

The first step is to ask yourself, what needs does the app meet? Who is it for? For end customers or business owners? Or maybe both. In order to answer all these questions you must consult from the best app development service providers and why not us? We will get to know and ask all the right questions, at the end of the conversation you will know exactly which application you want to develop and how we can help you. The next step is the start-up conversation, the beginning of our joint process together, we will introduce you to all the factors involved in the project and we will get to know you more in depth, we will present the project and schedules, it is your time to recruit all people involved. If you want to target both smartphone users and designed your app according to iOS devices then as the best iOS app development service in India, we assure you that the effort will pay off and in a few months you will have an invested and professional app.

Android app development service in India

What are the Major Advantages of Android App Development over IOS?

Software Developer – In today’s mobile and hassle-free era, it’s not wrong if applications that support business activities are a top priority for technology users, be it Android or iOS.

There are a lot of mobile devices used by the customer, but which platform will be best suited for your business / business model? How will you benefit your business and whether the platform can keep up with the latest industry updates and trends?

Both Android and Apple have their own advantages and disadvantages. Android allows you to reach a large audience, while Apple’s audience is more committed and loyal. For large businesses with adequate resources, they can grow on more platforms. But for small and medium enterprises, it is an important question that can decide whether your business will succeed or fail.

With more than 80% market share, Android devices are expected to dominate the global smartphone world in the years to come and beyond. One of the biggest benefits of Android app development  service in India that is, you can easily build, test, evaluate, and improve the application. This is one of the ideal ways to create business apps and improve user engagement. Also for this reason, so many people still choose to study android programming nowadays.

Flexibility in its use is the main point of the rapid development of mobile applications today.

The Android and iOS operating systems are the two most widely used mobile operating systems today. For businesses and businesses, of course it would be better if they made a business mobility application for the two operating systems, but not all business actors and businesses have sufficient budget for that because making these applications requires a large budget.

Android or iOS?

Before determining which operating system to use, it’s good if you already know the advantages and disadvantages of each of these operating systems.

Below are the advantages and disadvantages of these two operating systems:

In terms of application development, Android applications are generally built with the Java and Kotlin programming languages.

Android app development service in India are usually also assisted by Advance Google Development Tools such as Android Jetpack , Firebase, and the Android SDK which are a collection of libraries, features, and guides to help developers build and develop Android applications more easily.

The advantages of the Android Operating System

  • Android developers receive increased access to feature development
  • Android developers can develop applications for various Android devices, because not only smartphones use this operating system, other devices such as smart watches, TVs and so on.
  • Android developers can use the Google Design Guidelines to design a user interface that is easy for users to understand.

Conclusion

In the end, before determining what application you will build, it’s a good idea to consider it first, because the two operating systems have their respective advantages and disadvantages. But you must consider that Android has a larger user base and its development is more cost effective. And don’t forget to base these considerations on budgets, business ideas, targets, and other things that are important.

iOS app development service in India

Developing Career In iOS App Development

Mobile app development has become a new need and demand in the industry. These days all the businesses and works are almost done through mobiles which makes the developer the demanded god. As spread and expanded are the services and their applications, the career for an app developer is equally in the craze. India has seen substantial development in IT, and demand even shot up with the persistent pandemic requiring every possible idea to be launched digitally. The iOS applications have successfully traversed many domains. So if you are eager for a job in an iOS app development service in India, there is a complete clean career flow to follow to reach your destination.

Understand The Job Role

The app developers differ in their functionalities based on different OS. Either way, their function involves developing customer-friendly app platforms.

  1. Creating an app involves the proper knowledge of the application’s purpose and the requested layout.
  2. The selection and expertise to use apt software and devices is a must to reach surety.
  3. The role extends to updating and the attentive modifications based on real-time changes. 
  4. The apps should be compatible with popular iOS using smartphones, watches and iPads.  

Path To Career

The companies aiming for iOS app development service in India have strict selection policies to fish out the gems among the lot. Thus, one can progressively develop and train with the ambition to reach the goals.

  1. Degree: A graduation degree in Computer Science or IT is desired for the software-centric job. It covers the basic required understanding for the job role.
  2. Extra Courses: The programming languages and application developments differ with apps and companies. Thus, the aspirants can choose to enrol in distant courses for separate coding and developing study. The courses are available online or offline through diploma certificates.
  3. Internships: Don’t haste from applying to internships in IT-based service companies. They train in the core developing and programming skills that enhance complete creativity.
  4. Attend Bootcamps: Bootcamps have proven fact for profitable practical exposure and practice in real-time practice. In addition, many companies like Microsoft, Google and Amazon hire candidates based on the Bootcamp certificates.
  5. Target Specific Skills: The iOS developers need special qualities in:
  6. Swift programming language with updates and amendments. It is the largely used backend programming for app development. Coding, adjusting interfaces and debugging requires language input. 
  7. The ULKit is the developer’s interface used for frontend applications along with the SQLite database applications.
  8. Xcode is an exclusively designed application for Apple developers to develop apps and web platforms.
  9. Networking, animations and image editing also contribute to the development. Clients always look for easy to navigate and attractive components in their products.

The job requirement isn’t analytical but also peruses soft skills and better communication. The interviews and selection also judge based on them.

It isn’t a fortnight’s miracle to develop an enhanced app or get placed in an established app development service. The progressive practice, and continuous updates with the latest techniques and languages help crack the code to build a robust career. 

img2-1.png

Case Study: The Bail Project

CASE STUDY: THE BAIL PROJECT

img2-1.png

Challange

 
 

The Bail Project™ National Revolving Bail Fund is a critical tool to prevent incarceration and combat racial and economic disparities in the bail system.

 

The Bail Project combats mass incarceration by disrupting the money bail system—one person at a time. This restores the presumption of innocence, reunites families, and challenges a system that criminalizes race and poverty. The Bail Project on a mission to end cash bail and create a more just, equitable, and humane pretrial system.

To combat mass incarceration The Bail Project has to set up and consume feeds from various jail & court websites to keep track of the next hearing date and also to follow up with the person in need to make sure his appearance in court.

What Complere did?

 
 
 
 
 
 
 
 
 
 
 

Complere approach to set up web scrapers and data warehouse solution on AWS Cloud to hold data from various jail & court websites, which support data discovery and follow-ups requirements to ensure the appearance of the concerned person in court. We have built & enhanced the scrappers specific to jail & court websites and set up the standardised staging and data warehouse layers to support data discovery.

The scope of the engagement included the following initiatives :

Build/enhance the node js scripts to scrape data from various Jail & Court websites and capture raw data in MongoDB.

Build the process to bring raw data from MongoDB to the staging area.


Build the process to transform the data identified in the staging area to populate data warehouse fact and dimension tables using PL/pgSQL

Build a process to bring messages from AWS DynamoDB and make them available in the data warehouse.


Build a process to capture leads, accounts, contacts, cases and other information from salesforce and make it available in the data warehouse.

Deploy ETL jobs on the AWS EC2 servers and set up a schedule to run the daily end-to-end process.


Setup the batch control to handle increment or full load specific to each source.



Setup Audit control system for each execution of process instance with the end status of SUCCESS or FAILED with an appropriate error message to resolve the failure.

Build the error logging and reject logging mechanism to identify and troubleshoot failures.

Complere solution benefited the The Bail Project in the following ways:

The centralized data warehouse solution helps The Bail Project by integration the data from various sources like the jail & court websites. Salesforce, and messages from DynamoDB to find & follow ups persons in need

The development of ETL in the Talend data integration tool & PL/pgSQL has reduced 70% efforts on the development side.




The easy to understand & extensible approach has benefited the project to self manage and extend the use case.



Summary

 

Complere provided The Bail Project with simple, easy to handle, high-performance ETL pipelines and data warehouse solution on AWS Cloud. With the new solution,  The Bail Project can take advantage of all consolidated data in the warehouse to identify the next needful person. Improved data quality and data availability would not only help them quickly realize high ROI on their investment but also enable them to end cash bail and create a more just, equitable, and humane pretrial system. Because bail is returned at the end of a case, donations to The Bail Project™ National Revolving Bail Fund can be recycled and reused to pay bail two to three times per year, maximizing the impact of every dollar

android app development service in india

How app development services are more reliable in India?

In the trending system, everyone favors generating an app as a primary one. Most height of organizations in society wishes to produce a portable app for each product. A sight to know that each and wherever the mobile application is assigned a significant role. App designing is nothing where they provide a set of software details as per the client’s terms. There are different types of android app development service in india. They are designed the software by the specialist developer-only then the application will be transferred by added users. They have a separate way of developing the software and discovering it.

What are the talents are to have an app developer?

First of all those, they ought to well experience about the technological abilities. Some of the following skills forapplication development have to perceive.

  • Well knowledge about the base
  • Familiar in programming
  • Expertise in different thoughts
  • conceding the terms

These are well essential in designing the software.

What are the benefits of developing an app?

Throughout the system, technology is implemented from banking to shopping malls. By using these technologies you will preserve your time. If you are existing in your home you will prefer the online platform for everything. These are all functions by the android app development service in indiaEverything has easy by utilizing these settings. The app developer builds software for the user’s requirements only so they can quickly develop the best thing in the environment.

There are numerous benefits of application designing in some particular fields. You will obtain more extra information about the thing by utilizing the app. These are user-friendly to the client to gain numerous data. In the enterprise field, it is the leading. You will obtain your report or date anywhere and whenever by using the app development.

Reason for uniqueness:

Assuming you are designing various applications for your business, for the easy approach you will get more consumers in a short period. Due to this desired outcome businessmen prefer the application for their product so most of them are turns into this side. It is worthwhile while utilizing it whenever. Not all they know about designing the app by themselves so you have to suggest a unique android app development service in India to get your app. They will make reliable and secure ones for business usage. There are different people who are starting to establish this different one hence it is the most needed one for the world. Within your budget you will create your app definitely will use for the people and business person too. Their reliable services provide advantages to the business. 

Summary:

Their services are trustworthy; they will design only the best app for the products. Most maximum people are utilizing these services for an efficient outcome. Finally, you pick the app development services for the business development doesn’t miss the opportunity for any reason. This is an extraordinary one and nothing can reestablish their services. The straightforward things it holds several benefits and advantages. 

meeting

Case Study: Textellent

CASE STUDY: TEXTELLENT

CTA-contact-us-new

Challange

 
 
 
 

Textellent offers a better way to engage and keep customers through text message marketing. To enable the Tax preparation services Textellent has to build the integration with different software such as Drake, Intuit ProSeries, Taxwise Desktop & Online, CrossLink Professional Tax Software, TaxWare, Tax Slayayer Pro, Lacerte Tax, Rig Tax and others.

 Textellent has to build at speed and scale to handle clients’ info uploads of Textellent customers from various Tax software, before the import files required to be validated and then processed.

What Complere did?

 
 
 
 
 
 
 
 
 
 
 

Textellent, a leader in Business Texting Solutions and named to one of the top spots by Software Advice (a Gartner company) in the 2020 year-end FrontRunners Report for SMS Marketing Software. Textellent provides a robust, flexible platform with rich out-of-the-box functionality and strong integration capabilities with Tax Softwares.

Complere approach to building real-time Integrations as microservices using Talend ESB for each Tax Softwares, where we have split the problem into master, branches and slave concept. Masters service is responsible to route the validation or process request of the requested software branch, Branches are where validation or processing is done of the client info and the slave job is common for all the branches which are responsible to load the transformed data into the database.

The scope of the engagement included the following initiatives :

Build the transformations from scratch specific to each Tax software.


Build a robust REST framework to take the request from the Textellent portal and send the same for validation and processing.

Build the REST validation framework which should validate the client info uploads from Textellent customers, response error and its description in a realtime.

Build the standard slave process to consume processed files produces by the branches.


Build the error logging and rejects logging mechanism to identify and troubleshoot failures.

Build a process to support both real-time client info validation and scheduled based client info processing

Complere solution benefited the Textellent in the following ways:

The generic framework in the system requires minimal customization, which ultimately saves development efforts and significantly reduces the cost of future implementation.

The Talend ESB tool based microservices reduced 70% efforts on the development side.




Using the Master, branches and Slave concept saved development costs by more than 60%




Real-time data validation solution increased the customer satisfaction score by identifying the mistakes in the client info uploads.


Summary

 

Complere provided Textellent with a simple, easy to handle and high-performance data transformation solution. With the new solution, Textellent can enable its customers to uploads their client info and manage the transformed data in a centralized location. Improved rate and quality of transformations would not only help them quickly realize high ROI on their investment but also enable them to acquire major customers with a large business base in the long run.  

board, chalk, business

Atlassian Bitbucket To AWS CodeCommit Using Bitbucket Pipelines

Hello everyone,

Today we going to learn about how we can sync the data from the bitbucket to the AWS CodeCommit .There can be cases we will be using Bitbucket from long time and now we have to shift ourself to the CodeCommit .So we will need to replicate our commits done till date in the bitbucket and will also need to do the sync on timely basis.To achieve our goal bitbucket pipeline will help us with it .Lets see how we can do it .

Short Description on steps we will be following

  • Creating a new and empty CodeCommit Repository where we are going to sync the data of the bitbucket repository
  • Creation of IAM Group which will have the access permissions which will allow us to commit the changes in the Codecommit repository
  • Creation of IAM  User through which we will commit the changes from the bitbucket to the CodeCommit
  • Creation of SSH Keys and adding in the Security Credentials of the user
  • Configure Bitbucket Pipelines which will help us to create the replication from the bitbucket repository to the CodeCommit and which will be helpful to maintain the sync on timely basis.

Procedure

  • Creation of CodeCommit RepositoryFirst we will create an empty repository by selecting the region where we want the CodeCommit repository to be .Following will be the steps to create a new repository
    • We will create an empty repository to commit the changes from Bitbucket
    • Open up AWS CodeCommit and select your region
    • Once you’ve created a repository, select the repository, click the “Connect” button, and choose the SSH option which we’ll be using later on, this is where you’ll find your connection information, and some instructions that you can refer back to later.

Creation of IAM Group

Here we will need to have the Permission to the user for the CodeCommit to commit the changes

  • Create a new IAM CodeCommit-Contributor
  • Assign the AWSCodeCommitPowerUser policy to this group

Creation of IAM  User

We will create a new user which will be helpful for us to get the data from the bitbucket to the CodeCommit

  • Create a new IAM user with a login of Bitbucket-User
  • assign the CodeCommit-Contributor group to it.
  • After creation we will add the SSH public key to the user which we will do below

Creation of SSH Keys and adding in the Security Credentials of the user

Access to CodeCommit repositories is provided by associating credentials or keys. In this case, we’re going to use SSH and generate public and private keys for use with the IAM user and Bitbucket Pipeline service.

To generate a new private and public key (Windows users, YMMV), we’ll open terminal and execute the following. We’re not going to provide a password here, just hit return when it asks.

To generate a new private and public key (Windows users, YMMV), we’ll open terminal and execute the following. We’re not going to provide a password here, just hit return when it asks.

ssh-keygen -f ~/.ssh/codecommit_rsa

  • This will generate 2 files, ~/.ssh/codecommit_rsa, which is the private key and ~/.ssh/codecommit_rsa.pub, which is the public key. Copy your public key to your clipboard:
pbcopy < ~/.ssh/codecommit_rsa.pub
or we can do is
sudo cat ~/.ssh/codecommit_rsa.pub and copy the contents in the clipboard
  • Open your IAM Bitbucket-User, and under “Security credentials”, click Upload SSH Key under “SSH keys for AWS CodeCommit”, and paste in your public key.
  • Once your public key is created, there will be an SSH key ID associated with it.
  • This will be used as your CodeCommit username when accessing repositories.

Set up Git and validate your connection

Let’s test the connection at this point to confirm that you’ve correctly associated your new key with the user, as well as validated that the user has the correct privileges in the CodeCommit profile assigned to the group. We’re going to use this same configuration later on with Bitbucket Pipelines, so keep it handy.

  • Create your ~/.ssh/config, and associate your IAM user’s SSH key ID and new private key with the CodeCommit hosts.Write the below details in the config file which we will create
Host git-codecommit.*.amazonaws.com
  User Your-IAM-SSH-Key-ID-Here [which is created in Security credentials when we uploaded the SSH key in iam user]
  IdentityFile ~/.ssh/codecommit_rsa
  • Now we will initialize the connection as below
ssh git-codecommit.us-east-1.amazonaws.com

The authenticity of host ‘git-codecommit.us-east-1.amazonaws.com (72.21.203.185)’ can’t be established.

RSA key fingerprint is SHA256:XXX/XXXXXX.
Are you sure you want to continue connecting (yes/no)? yes

Warning: Permanently added ‘git-codecommit.us-east-1.amazonaws.com,72.21.203.185’ (RSA) to the list of known hosts.

We should get the below response :

  • You have successfully authenticated over SSH. You can use Git to interact with AWS CodeCommit. Interactive shells are not supported.Connection to git-codecommit.us-east-1.amazonaws.comclosed by remote host.

Configure Bitbucket Pipelines

  • In order to use Bitbucket Pipelines, it needs to be enabled for the repository first. Under your repository settings, choose Pipelines and enable pipelines in bitbucket.
  • Now Pipelines is enabled, and before configuring that bitbucket-pipelines.yml file, lets initialize some Pipelines environment variables.
  • Under your repository settings, choose Repository Variables under Pipelines. We’re going to create 5 environment variables as below.

Following are the variables we will assign

  • CodeCommitConfig: The base64 encoded version of the SSH config we added to our ~/.ssh/config earlier that specifies the Host, User and IdentityFile.
    • We can create the base64 encoding below
cat ~/.ssh/config | base64 -w 0
  • CodeCommitHost: The host and region of your CodeCommit instance
  • CodeCommitKey: The base64 encoded version of your SSH private key that we generated (node that it’s hidden and encrypted in the above screenshot because Secured was selected, make sure you do this as well).We can create base4 encoding like
cat ~/.ssh/codecommit_rsa | base64 -w 0
  • CodeCommitRepo: The host, region and repository path of your repository.
  • CodeCommitUser: The SSH key ID associated with the public key on your AWS IAM user.[This is the SSH keyID which we will get in the Security Credentials in the IAM]
  • Lets create that bitbucket-pipelines.yml file, either add it using your favourite editor, or click “Configure bitbucket-pipelines.yml” and edit it directly on bitbucket.org.
pipelines:
  default:
    – step:
        script:
          – echo $CodeCommitKey > ~/.ssh/codecommit_rsa.tmp
          – base64 -d ~/.ssh/codecommit_rsa.tmp > ~/.ssh/codecommit_rsa
          – chmod 400 ~/.ssh/codecommit_rsa
          – echo $CodeCommitConfig > ~/.ssh/config.tmp
          – base64 -d  ~/.ssh/config.tmp > ~/.ssh/config
          – set +e
          – ssh -o StrictHostKeyChecking=no $CodeCommitHost
          – set -e
          – git remote add codecommit ssh://$CodeCommitRepo
          – git push codecommit $BITBUCKET_BRANCH
  • Below is the details of the pipeline script which we have created
  • Creates temporary files for $CodeCommitKey and $CodeCommitConfig then decodes them into place.
  • Adjusts the permissions on your primary key (some SSH clients require more secure privileges on this file)
  • Initializes the SSH connection to the CodeCommit host. It’s worth noting here that this command will “appear to fail”, so we need to disable error checking (set +e) on this script and let it fail silently and then re-enable error checking (set -e). -o StrictHostKeyChecking=no will prevent the service from needing to manually accept the remote host.
  • Add the CodeCommit repository as a remote and push the current ($BITBUCKET_BRANCH) branch

Notes

  • We will also require CodeCommit Repository as empty everytime
code, html, digital

Serverless Lambda Function For Talend Jobs

Learning about Talend and AWS is always fun and the way it is interacted is also full of fun. In this post we will see how run the talend job by deploying them in lambda function and run the lambda function using Serverless framework.

Lets get started with the prerequisites which are very important to start the topic:

Prerequisites

Installation of Node JS :

Lets Download the Node JS binaries from (https://nodejs.org/en/) site and install.
Node JS installation will install both node JS runtime and npm (node package manager)
NPM is used to install packages.

For linux we have to use command as

<code

yum install-y gcc-c++make
curl -sL https://rpm.nodesource.com/setup_6.x |sudo-Ebash
yum install nodejs
node -v

For windows we have to install like : https://nodesource.com/blog/installing-nodejs-tutorial-windows/

Installation of Apache Maven:

As we are done with the Nodejs installation we will start with apache maven

Following are the ways how we can install apache maven :

In Windows :

  • Go to the link :https://maven.apache.org/download.cgi unzip it and add the bin path to the environment variables like :
    • PATH=C:\apache-maven-3.6.0-bin\apache-maven-3.6.0\bin
    • We can verify if it is installed by checking mvn -version [Which will give us the version of the maven which is installed.]

In Linux :

  • wget http://mirror.olnevhost.net/pub/apache/maven/maven-3/3.0.5/binaries/apache-maven-3.0.5-bin.tar.gz
  • Run the wget command from the dir you want to extract maven too.
  • run the following to extract the tar,
    tar xvf apache-maven-3.0.5-bin.tar.gz
  • move maven to /usr/local/apache-maven
    mv apache-maven-3.0.5  /usr/local/apache-maven
  • Next add the env variables to your ~/.bashrc fileexport M2_HOME=/usr/local/apache-maven
    export M2=$M2_HOME/bin
    export PATH=$M2:$PATH
  • Execute these commands
    source ~/.bashrc
  • Verify everything is working with the following commandmvn -version

Installation of Oracle JDK

Serverless configuration

We will need to install serverless and add our credentials in config which will help us to connect with the aws

  1. Run the below command to install Serverless globallyserverless-stack-output is a plugin, aws-sdk is used to call Batch jobs and to install other dependencies using npm installnpm install -g serverlessnpm install
  2. serverless config credentials
sudo serverless config credentials –provider aws –key <ACCESS_KEY> –secret <SECRET_KEY>

Serverless installation of the AWS Lambda function

This will create an AWS Lambda function for the talend job using the serverless.

  • First we will make empty directory named as : serverless-lambda-talendjobname
  • Now in Visual code we will browse to the folder as cd serverless-lambda-talendjobname
  • Now we will create a serverless java maven template as
     serverless create –template aws-java-maven
  • Now we have to install the talend related directories for that we have one zip attached as Supporting_Talend_Jobs_For Serverless
  • This zip will help us to install all the lib files and will also generate the pom.xml for us which we can use it for the serverless_Project.
  • Following will be the generic folder structure which we will follow :
  • Following are the details of all the folders and zips present in the above image
    • serverless_project : We can give the generic name to the serverless_project which will be parent directory and where we will place all the related folders and zips .
    • code : This will contain the serverless code which we will install and deploy
    • lib : This will contain all the libs which are present and which needs to do mvn install
    • Supporting_Talend_Job_For_Serverless : This is the attached zip which is used to install the libs of the talend projects
    • TalendProject : This is the talend project for which we have to create the lambda function
  •  Working Procedure of Supporting_Talend_Job_For_Serverless :
    • 1st we have to unzip the Supporting_Talend_Job_For_Serverless and also unzip the TalendProject
    • TalendProject have jars at 2 places :
      1. TalendProject\lib
      2. TalendProject\TalendProject\lib
      3. We will place this jars in the lib folder which is shown in above image
    • Then we will run the bat file present in the directory D:\serverless_project\Supporting_Talend_Job_For_Serverless_0.1\Supporting_Talend_Job_For_Serverless\Supporting_Talend_Job_For_Serverless_run.bat which will do all the installation of the jars and also will generate the pom.xml file named as pom_generated_by_talend.xml
  • We will now replace the pom.xml file created in the code directory from the template to the one which is generated by talend
  • We will change the Handler.java as per the code like below
public ApiGatewayResponse handleRequest(Map<String, Object> input, Context context) {
        LOG.info(“received: {}”, input);
        testproject.newjob_0_1.newjob t2 = new testproject.newjob_0_1.newjob();
        String[] context2 ={};
        t2.runJob(context2);
        Response responseBody = new Response(“Success”,input);
        return ApiGatewayResponse.builder()
            .setStatusCode(200)
            .setObjectBody(responseBody)
            .setHeaders(Collections.singletonMap(“X-Powered-By”, “AWS Lambda & serverless”))
            .build();
}

Now we will try to install the dependencies as :

mvn clean install

When the build will be success we will deploy the sls as :

sls deploy

We can now check the function by invoking it as :

serverless invoke –function <functionname> -l

And we are done if we get any error then we can check the above steps .

pixel image

ETL With Airflow

Airflow is a platform to programmatically author, schedule and monitor workflows.

We use airflow to author workflows as directed acyclic graphs (DAGs) of tasks. The airflow scheduler executes your tasks on an array of workers while following the specified dependencies, we have heard about airflow but we never knew how to get it working for the talend jobs and it will be very easy as we will have the UI for scheduling and monitoring the working flow of the talend jobs.

Lets check our To Do to achieve the goal

  1. Launching the instance of the Ec2 : We will be launching the ubuntu server for the installation of the airflow and also for copying the talend jobs in the server
  2. Installing Airflow in Ec2 instance : We will follow the steps for the installation of the airflow and get the webserver of the airflow working
  3. Adding of the talend job and creating DAGs file

Launching an ec2 instance in aws.

We will launch ubuntu 16.04 instance for airflow

Adding of Airflow in Ubuntu Server

Call Now Button