{"id":101,"date":"2023-01-23T11:01:38","date_gmt":"2023-01-23T11:01:38","guid":{"rendered":"https:\/\/keyshell.net\/blog\/?p=101"},"modified":"2023-01-23T11:01:38","modified_gmt":"2023-01-23T11:01:38","slug":"how-to-integrate-google-cloud-build-with-jfrog-artifactory","status":"publish","type":"post","link":"https:\/\/keyshell.net\/blog\/2023\/01\/23\/how-to-integrate-google-cloud-build-with-jfrog-artifactory\/","title":{"rendered":"How to Integrate Google Cloud Build with JFrog Artifactory"},"content":{"rendered":"

In this blog we describe how to Integrate Google Cloud Build with Jfrog Artifactory.We will build a sample containerized application that pulls dependencies from Artifactory, with Maven and Java as our sample package and language.<\/span><\/p>\n

JFrog is a software company that provides a platform for managing and distributing software artifacts. The company’s main product, JFrog Artifactory, is a universal artifact repository that supports all major packaging formats, including Docker, Maven, npm, and NuGet. JFrog also offers other tools such as JFrog Xray and JFrog Mission Control for managing and monitoring software artifacts across an organization.<\/span><\/p>\n

What is an Artifact ?<\/b>
\n<\/span>An artifact is a by-product of software development. It\u2019s anything that is created so a piece of software can be developed. This might include things like data models, diagrams, setup scripts etc.<\/span><\/p>\n

\u201cArtifact\u201d is a pretty broad term when it comes to software development. Most pieces of software have a lot of artifacts that are necessary for them to run. Some artifacts explain how a piece of software is supposed to work, while others actually allow that program to run.<\/span><\/p>\n

Artifacts are important to hold onto throughout the development process of any piece of software, and even long after.<\/span><\/p>\n

Without each and every artifact, it can make developing a piece of software much more difficult over time. This is especially true if development switches hands. When a new developer is put on a project, one of the first things they\u2019ll want to do is go through the artifacts to get an idea of how the software works.<\/span><\/p>\n

If an artifact is missing, that leaves a developer in the dark. This is why most artifacts are kept in a repository. This lets relevant developers access the artifacts at any time, all from one place.<\/span><\/p>\n

What is Artifactory ?<\/b><\/p>\n

Artifactory is Jfrog\u2019s tool which acts as an artifact repository.It\u2019s a place to store all your Binaries,Builds & Metadata.<\/span><\/p>\n

Prerequisites:<\/i><\/b>
\n<\/i><\/b>Google Cloud Account : Should Enable Google Cloud Build<\/span><\/i><\/p>\n

JFrog Account :We used 30 days trial from Google Cloud\u2019s MarketPlace (<\/span><\/i>https:\/\/console.cloud.google.com\/marketplace\/product\/jfrog\/jfrog-pro-team-saas<\/span><\/i><\/a>)<\/span><\/i><\/p>\n

First,Let\u2019s Install and Setup Google Cloud CLI.We are using Ubuntu 20.04 LTS (Debian).You can skip this step if you have source code in <\/span>Google Cloud Shell<\/span><\/i> and edit using <\/span>Google Cloud Editor<\/span><\/i>.<\/span><\/p>\n

1)To download the Linux 64-bit archive file, at the command line, run:<\/span><\/p>\n

$ curl -O https:\/\/dl.google.com\/dl\/cloudsdk\/channels\/rapid\/downloads\/google-cloud-cli-413.0.0-linux-x86_64.tar.gz<\/span><\/i><\/p><\/blockquote>\n

2)Extract the contents of the file to any location on your file system (preferably your Home directory)<\/span><\/p>\n

$ tar -xf google-cloud-cli-413.0.0-linux-x86.tar.gz<\/em><\/p><\/blockquote>\n

3)Run the installation script from the root of the folder you extracted to using the following command:<\/span><\/p>\n

$ .\/google-cloud-sdk\/install.sh<\/span><\/i><\/p><\/blockquote>\n

4)<\/span>To initialize the gcloud CLI, run:<\/span><\/p>\n

$ .\/google-cloud-sdk\/bin\/gcloud init<\/span><\/i><\/p><\/blockquote>\n

5)After initialize,add to path :<\/span><\/p>\n

$ export PATH=$PATH:\/home\/user\/google-cloud-sdk\/bin<\/span><\/i><\/p><\/blockquote>\n

After this we can run gcloud CLI commands directly.<\/span>
\n<\/span>Eg :<\/span> $ gcloud info<\/span><\/i><\/p>\n

For reference for setting up Google Cloud CLI refer : <\/span>https:\/\/cloud.google.com\/sdk\/docs\/install-sdk#linux<\/span><\/a><\/p>\n

Next,we can clone the source code from github.<\/span><\/p>\n

$ git clone https:\/\/github.com\/Keyshelltechs\/Keyshell_Jfrog.git<\/span><\/i><\/p><\/blockquote>\n

Folder Structure :\u00a0<\/span><\/p>\n

\ud83d\udcc1examples<\/span><\/p>\n

\ud83d\udcc3DockerFile<\/span><\/p>\n

\ud83d\udcc3cloudbuild.yaml<\/span><\/p>\n

First we have to build a Maven Image that includes JFrog CLI. For that we can use <\/span>cloudbuild.yaml<\/span><\/i> and <\/span>DockerFile<\/span><\/i>.<\/span><\/p>\n

Snippet of <\/span>cloudbuild.yaml <\/span><\/i><\/p>\n

- name: 'gcr.io\/cloud-builders\/docker'\r\n  args:\r\n  - 'build'\r\n  - '--build-arg=BASE_IMAGE=gcr.io\/cloud-builders\/mvn:3.5.0-jdk-8'  \r\n  - '--tag=gcr.io\/$PROJECT_ID\/java\/jfrog:1.54.1'\r\n  - '.'\r\n  wait_for: ['-']<\/code><\/pre>\n
gcr.io\/cloud-builders is a public container registry.<\/span>
\nHere we use an image of Maven(<\/span>mvn:3.5.0-jdk-8<\/span><\/i>) from gcr.io\/cloud-builders as base image to build Maven Image including JFrog CLI.You can see the image here<\/a>.
\nDockerFile<\/i><\/span><\/p>\n
ARG BASE_IMAGE=gcr.io\/${PROJECT_ID}\/mvn:3.5.0-jdk-8\r\nFROM ${BASE_IMAGE}\r\n\r\nARG JFROG_CLI_VERSION=1.54.1\r\n\r\n# PR submitted to download versioned JFrog CLI\r\n\r\nRUN apt-get update -qqy && apt-get install -qqy curl \\\r\n  && cd \/tmp \\\r\n  && curl -fL https:\/\/getcli.jfrog.io | sh \\\r\n  && mv jfrog \/usr\/bin\/ \\\r\n  && apt-get remove -qqy --purge curl \\\r\n  && rm \/var\/lib\/apt\/lists\/*_*\r\n\r\nENTRYPOINT [\"jfrog\"]<\/code><\/pre>\n
Here we use this DockerFile to build the final image by adding JFrog CLI to the base image.<\/span><\/p>\n
So the final image will be tagged as <\/span>gcr.io\/$PROJECT_ID\/java\/jfrog:1.54.1<\/span><\/i><\/p>\n
Now let\u2019s build this image using Google Cloud Shell\/Google Cloud CLI.<\/span><\/p>\n
To build using Google Cloud Shell use this command : <\/span><\/p>\n
$ gcloud builds submit –config=cloudbuild.yaml –project=your_google_project_id .<\/span><\/i><\/p><\/blockquote>\n
To build using Google Cloud CLI use this command : <\/span><\/p>\n
$ gcloud builds submit –config=cloudbuild.yaml .<\/span><\/i><\/p><\/blockquote>\n
If the build is successful we can view the image in your GCP\u2019s container registry.<\/span><\/p>\n
Container Registry > Images > Java > JFrog<\/span><\/p>\n
To check build results navigate to Cloud Build > History<\/span><\/p>\n
Now we have built a Maven Image that includes JFrog CLI.We have to configure it to point it to Jfrog Artifactory.<\/span><\/p>\n
For that first we have to create a <\/span>Virtual Snapshot Repository<\/span><\/i> and a <\/span>Virtual Release Repository <\/span><\/i>in Jfrog.<\/span><\/p>\n
For creating these we have to login to our Jfrog Account.<\/span><\/p>\n
After Logging in, let\u2019s take a look on how we can create a <\/span>Virtual Release Repository.<\/span><\/i><\/p>\n
For creating a Virtual Repository , first we have to create a local and remote repository.<\/span><\/p>\n
Before that let\u2019s check what a local,remote and virtual repository is.<\/span><\/p>\n
Local Repository:<\/b> In JFrog, a local repository is a repository that is stored on the same machine as the JFrog Artifactory instance. It functions as a cache of all the artifacts that have been downloaded from remote repositories, and as a target for deploying artifacts that have been built locally.The local repository allows for faster access to frequently used artifacts and can also be used to manage internal artifacts that should not be exposed to external systems.<\/span><\/p>\n
 <\/p>\n
Remote Repository: <\/b>In JFrog, a remote repository is a repository that is stored on a different machine or in a different location than the JFrog Artifactory instance. It generally refers to external repositories from where the artifacts are downloaded. A remote repository can be located in a different Artifactory server, or it can be a repository hosted by a third-party provider, such as Maven Central or JCenter. Remote repositories provide a way to access external artifacts that are needed for a build, and to share artifacts that have been deployed to Artifactory with other teams or systems. When a client requests an artifact that is not present in any of its local repositories, Artifactory will check the remote repositories in the order they are defined in the system, and will download the artifact from the first repository that contains it.<\/span><\/p>\n
Virtual Repository: <\/b>In JFrog, a virtual repository is a virtual aggregation of multiple local, remote and other virtual repositories. It allows you to access all of your repositories as if they were a single, unified repository. This makes it easy to manage and access multiple repositories from a single URL, and it also allows you to easily switch between different sets of repositories depending on your needs.<\/span><\/p>\n
For example, you can use a virtual repository to aggregate all of your local, remote and other virtual repositories into a single URL, and then use that URL as the repository in your build tool’s configuration. This allows you to switch between different sets of repositories without changing your build configuration. Additionally, it allows you to manage access and permissions for a group of repositories together, and also search for artifacts across all repositories in the virtual repository.<\/span><\/p>\n
 <\/p>\n
Virtual repositories can be very useful to organize your artifacts and make it easier to manage access and permissions for different projects, teams or environments.<\/span><\/p>\n
 <\/p>\n
Now let\u2019s create Local,Remote and Virtual Repository<\/span><\/p>\n
To create Local Repository :\u00a0<\/span><\/strong><\/p>\n
Select Administration > Repositories > Repositories > Add Repositories > Local Repository<\/span><\/p>\n
Then select \u201cMaven\u201d as Package Type.<\/span><\/p>\n
\"\"<\/p>\n
 <\/p>\n
Add \u201ckeyshell-maven-local\u201d as Repository Key and click on \u201cCreate Local Repository\u201d<\/span><\/p>\n
\"\"<\/p>\n
To create Remote Repository :\u00a0<\/strong><\/span><\/p>\n
Select Administration > Repositories > Repositories > Add Repositories > Remote Repository<\/span><\/p>\n
Then select \u201cMaven\u201d as Package Type.<\/span><\/p>\n
Add \u201ckeyshell-maven-remote\u201d as Repository Key.<\/span><\/p>\n
Add \u201c<\/span>https:\/\/jcenter.bintray.com<\/span><\/a>\u201d as URL. (JCenter is a public, Bintray-hosted repository that is used to store and distribute Java and Android libraries. JCenter is maintained by JFrog)<\/span><\/p>\n
Now click on \u201cCreate Remote Repository\u201d<\/span><\/p>\n
\"\"<\/p>\n
To create Virtual Repository :\u00a0<\/strong><\/span><\/p>\n
Select Administration > Repositories > Repositories > Add Repositories > Virtual Repository<\/span><\/p>\n
Then select \u201cMaven\u201d as Package Type.<\/span><\/p>\n
Add \u201ckeyshell-maven-virtual\u201d as Repository Key.<\/span><\/p>\n
\"\"<\/p>\n
Here we have the option to select from our local,remote and other virtual repositories.<\/span><\/p>\n
Now select \u201ckeyshell-maven-local\u201d and \u201ckeyshell-maven-remote\u201d from the list in the same order.<\/span><\/p>\n
Also select \u201ckeyshell-maven-local\u201d as Default Deployment Repository.<\/span><\/p>\n
Now click on \u201cCreate Virtual Repository\u201d.<\/span><\/p>\n
\"\"<\/p>\n
Now we have created a <\/span>Virtual Release Repository <\/span><\/i>in Jfrog.Repeat the same steps to create <\/span>Virtual Snapshot Repository.<\/span><\/i><\/p>\n
Now let\u2019s go inside our \u201cexamples\u201d folder.<\/span><\/p>\n
\"\"<\/p>\n
Here cloudbuild.yaml file is used to Configure JFrog CLI to point to JFrog Artifactory,Build a sample maven project and then containerize java app.<\/span><\/p>\n
Snippet of <\/span>cloudbuild.yaml<\/span><\/i><\/p>\n
# Configure JFrog CLI to point to JFrog Artifactory<\/span>\r\n\r\n- <\/span>name<\/span>: <\/span>'docker.bintray.io\/google-cloud-builder\/java\/jfrog:0.1'<\/span>\r\n\r\n\u00a0\u00a0<\/span>entrypoint<\/span>: <\/span>'bash'<\/span>\r\n\r\n\u00a0\u00a0<\/span>args<\/span>: [<\/span>'-c'<\/span>, <\/span>'jfrog rt c rt-mvn-repo --url=https:\/\/[ARTIFACTORY-URL]\/artifactory --user=[ARTIFACTORY-USER] --password=[ARTIFACTORY-PASSWORD OR ARTIFACTORY IDENTITY TOKEN]'<\/span>]<\/span>\r\n\r\n\u00a0\u00a0<\/span>dir<\/span>: <\/span>'maven-example'<\/span>\r\n\r\n# Build a sample maven project<\/span>\r\n\r\n- <\/span>name<\/span>: <\/span>'gcr.io\/$PROJECT_ID\/java\/jfrog'<\/span>\r\n\r\n\u00a0\u00a0<\/span>args<\/span>: [<\/span>'rt'<\/span>, <\/span>'mvn'<\/span>, <\/span>\"clean install\"<\/span>, <\/span>'config.yaml'<\/span>, <\/span>'--build-name=mybuild'<\/span>, <\/span>'--build-number=$BUILD_ID'<\/span>]<\/span>\r\n\r\n\u00a0\u00a0<\/span>dir<\/span>: <\/span>'maven-example'<\/span>\r\n\r\n# Containerize java app<\/span>\r\n\r\n- <\/span>name<\/span>: <\/span>'gcr.io\/cloud-builders\/docker'<\/span>\r\n\r\n\u00a0\u00a0<\/span>args<\/span>:<\/span>\r\n\r\n\u00a0\u00a0- <\/span>'build'<\/span>\r\n\r\n\u00a0\u00a0- <\/span>'--tag=gcr.io\/$PROJECT_ID\/java-app:${BUILD_ID}'<\/span>\r\n\r\n\u00a0\u00a0- <\/span>'.'<\/span>\r\n\r\n\u00a0\u00a0<\/span>dir<\/span>: <\/span>'maven-example'<\/span><\/code><\/pre>\n
Here <\/span>docker.bintray.io\/google-cloud-builder\/java\/jfrog:0.1 <\/span><\/i>is Jfrog Artifactory image which is used to Configure JFrog CLI to point to the JFrog Artifactory.<\/span><\/p>\n
We have to update this cloudbuild.yaml with actual values of <\/span>[ARTIFACTORY-USER]<\/b>,<\/span> [ARTIFACTORY-URL]<\/b> and <\/span>[ARTIFACTORY-PASSWORD OR ARTIFACTORY IDENTITY TOKEN]<\/b><\/p>\n
ARTIFACTORY IDENTITY TOKEN<\/b> can be generated after logging to Jfrog.<\/span><\/p>\n
(Click on the user button on top right corner > Edit Profile > Unlock > Generate Identity Token)<\/span><\/p>\n
 <\/p>\n
The files required to build a sample maven project is under the \u201cmaven-example\u201d folder.We have to edit the \u201cconfig.yaml\u201d file in this directory.<\/span><\/p>\n
config.yaml<\/span><\/p>\n
version<\/span>: <\/span>1<\/span>\r\n\r\ntype<\/span>: <\/span>maven<\/span>\r\n\r\nresolver<\/span>:<\/span>\r\n\r\n\u00a0\u00a0<\/span>snapshotRepo<\/span>: <\/span>keyshell-snapshot-virtual<\/span>\r\n\r\n\u00a0\u00a0<\/span>releaseRepo<\/span>: <\/span>keyshell-maven-virtual<\/span>\r\n\r\n\u00a0\u00a0<\/span>serverID<\/span>: <\/span>rt-mvn-repo<\/span>\r\n\r\ndeployer<\/span>:<\/span>\r\n\r\n\u00a0\u00a0<\/span>snapshotRepo<\/span>: <\/span>keyshell-snapshot-virtual<\/span>\r\n\r\n\u00a0\u00a0<\/span>releaseRepo<\/span>: <\/span>keyshell-maven-virtual<\/span>\r\n\r\n\u00a0\u00a0<\/span>serverID<\/span>: <\/span>rt-mvn-repo<\/span><\/code><\/pre>\n
Update the values of <\/span>snapshotRepo<\/span><\/i> and <\/span>releaseRepo <\/span><\/i>with the Jfrog\u2019s virtual repositories which we have created earlier.<\/span><\/p>\n
Dockerfile<\/span><\/i> in this directory is used to containerize the java app.<\/span><\/p>\n
pom.xml<\/span><\/i> file is used in Apache Maven-based Java projects.It includes information such as the project’s dependencies on other libraries, the build plugins that should be used, and the project’s version number. This file is used by Maven to build and manage the project.<\/span><\/p>\n
src <\/span><\/i>directory contains the java program to display \u201cHello World\u201d.<\/span><\/p>\n
Now let\u2019s build this cloudbuild.yaml (inside examples folder) using Google Cloud Shell\/Google Cloud CLI.<\/span>
\nTo build using Google Cloud Shell use this command :<\/span><\/p>\n
$ gcloud builds submit –config=cloudbuild.yaml –project=your_google_project_id .<\/span><\/i><\/p><\/blockquote>\n
To build using Google Cloud CLI use this command :<\/span><\/p>\n
$ gcloud builds submit –config=cloudbuild.yaml .<\/span><\/i><\/p><\/blockquote>\n
If the build is successful we can view the image in your GCP\u2019s container registry.<\/span><\/p>\n
Container Registry > Images > java-app<\/span><\/p>\n
To check build results navigate to Cloud Build > History<\/span><\/p>\n
Once the app is containerized, it can be deployed on GKE or any other compute target.<\/span><\/p>\n
 <\/p>\n
We can view the Artifacts in Jfrog.<\/span><\/p>\n
Application > Artifactory> Artifacts<\/span><\/p>\n
\"\"<\/p>\n
Now we can run the containerized java app from GCP.<\/span><\/p>\n
Container Registry > Images > java-app > Select Image > Show Pull Command<\/span><\/p>\n
\"\"<\/p>\n
Let\u2019s try running our container image in a cloud shell \/ VM .<\/span><\/p>\n
For that we can use these docker commands.<\/span>
\n<\/span>To pull docker image of our containerized java app from the registry<\/span><\/p>\n
$ docker pull your_image_name<\/span><\/i><\/p><\/blockquote>\n
To view the images<\/span><\/p>\n
$docker images<\/span><\/i><\/p><\/blockquote>\n
To run the container<\/span><\/p>\n
$docker run -it image_name<\/span><\/i><\/p><\/blockquote>\n
\"\"<\/p>\n
\u201cHello World!\u201d is our expected output.<\/span><\/p>\n
If you have any queries contact us at \ud83d\udcf2 +91-81295 71359 or email us at support@keyshell.net<\/span><\/p>\n","protected":false},"excerpt":{"rendered":"
In this blog we describe how to Integrate Google Cloud Build with Jfrog Artifactory.We will build a sample containerized application that pulls dependencies from Artifactory, with Maven and Java as our sample package and language. JFrog is a software company that provides a platform for managing and distributing software artifacts. The company’s main product, JFrog […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"om_disable_all_campaigns":false,"pagelayer_contact_templates":[],"_pagelayer_content":"","footnotes":""},"categories":[1],"tags":[],"class_list":["post-101","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"_links":{"self":[{"href":"https:\/\/keyshell.net\/blog\/wp-json\/wp\/v2\/posts\/101","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/keyshell.net\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/keyshell.net\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/keyshell.net\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/keyshell.net\/blog\/wp-json\/wp\/v2\/comments?post=101"}],"version-history":[{"count":64,"href":"https:\/\/keyshell.net\/blog\/wp-json\/wp\/v2\/posts\/101\/revisions"}],"predecessor-version":[{"id":288,"href":"https:\/\/keyshell.net\/blog\/wp-json\/wp\/v2\/posts\/101\/revisions\/288"}],"wp:attachment":[{"href":"https:\/\/keyshell.net\/blog\/wp-json\/wp\/v2\/media?parent=101"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/keyshell.net\/blog\/wp-json\/wp\/v2\/categories?post=101"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/keyshell.net\/blog\/wp-json\/wp\/v2\/tags?post=101"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}