Can you use the server free function of puppeter to build a linked browser

Posted by 2tonejoe on Wed, 09 Mar 2022 09:14:40 +0100

In this tutorial, we will learn how to deploy server functionality next to the front-end application and create an API to generate images and get metadata from links.

With serverless functionality, developers can create and implement modern features and functions in their applications without the pain of setting up and managing back-end servers. These functions are hosted and deployed by cloud computing companies.

Netlify functions make it easy for applications hosted on netlify to create and deploy serverless functions.

precondition

To learn this tutorial, you need to be familiar with JavaScript, Vue JS, Git, GitHub and Netlify. You should also have a text editor (such as VS Code) with vector (for IntelliSense) installed and the latest version of Node installed on your machine. They can install Node here. You can check your identity and Node version by running the command in the terminal. node -v

You should also have an account on Netlify. If you haven't, you can create one.

What are we building

To show how to easily set up serverless functionality using a front-end application, we will build an application with a custom link previewer component.

This component sends a request with a URL to our serverless function. Then, the function uses the puppeter to obtain metadata from the target site using the URL and generate a screenshot of the site.

This function sends metadata and screenshots back to our front-end components to display them as link previews in the application.

This is a link to the sample project deployed on Netlify. This is GitHub Repo following.

Creating and setting up Vue applications

We will use the Vue CLI to create a Vue 3 application. We will also install and set up Tailwind CSS, a utility first CSS framework that provides classes that we can use for our applications without writing a lot of custom CSS.

Installing and setting up Vue

In order to quickly build Vue applications, we will use Vue CLI. To install Vue CLI, run:

npm install -g @vue/cli

After installing the CLI, we can create the project by running the following command:

vue create link-previewer

This will prompt us to select a preset for our installation. We will select "manual selection function" so that we can select the function we need. Here are my options:

Please pick a preset: Manually select features
? Check the features needed for your project: Choose Vue version, Babel, PWA, Router, Vuex, Linter
? Choose a version of Vue.js that you want to start the project with: 3.x
? Use history mode for router? (Requires proper server setup for index fallback in production) Yes
? Pick a linter / formatter config: Basic
? Pick additional lint features: Lint on save
? Where do you prefer placing config for Babel, ESLint, etc.? In dedicated config files

After selecting these options, we will be asked if we want to save the options as presets for later use. Select Y (yes) or N (no) and continue with the installation.

Run CD link viewer to enter the newly created project.

Installing and setting up Tailwind CSS

To install Tailwind, we will use the PostCSS 7 compatibility build because Tailwind relies on PostCSS 8 -- which Vue 3 does not support at the time of writing. Uninstall any previous Tailwind installation and reinstall the compatibility build:

npm uninstall tailwindcss postcss autoprefixer
npm install -D tailwindcss@npm:@tailwindcss/postcss7-compat postcss@^7 autoprefixer@^9

Create Tailwind profile

Next, generate tailwind config. JS and postcss config. JS file:

npx tailwindcss init -p

tailwind. config. The smallest JS file will be created in the root directory of this project.

Configure Tailwind to remove styles not used in production

In this Tailwind config. JS file, purge uses the path configuration options of all pages and components so that Tailwind can shake the unused styles in the production build:

// ./tailwind.config.js
module.exports = {
  purge: ['./index.html', './src/**/*.{vue,js,ts,jsx,tsx}'],
  ...
}

Include Tailwind in CSS file

Create/ src/assets/css/main.css file and use the @ Tailwind command to include the base, components and utilities styles of Tailwind:

/* ./src/assets/css/main.css */
@tailwind base;
@tailwind components;
@tailwind utilities;

body{
  @apply bg-gray-50;
}

Tailwind will exchange these instructions with all styles generated by its configuration based design system at build time.

Finally, make sure to import CSS into the file/ src/main. JS file:

// ./src/main.js
import { createApp } from 'vue'
import App from './App.vue'
import './registerServiceWorker'
import router from './router'
import store from './store'
import './assets/css/main.css'

createApp(App).use(store).use(router).mount('#app')

That's it. We can run our server:

npm run serve

Now the application is running. If we go to the URL provided, we should see Vue's default demo application and Tailwind's pre check basic style has been applied.

Install Tailwind CSS IntelliSense extension

For a smoother development experience, please install the Tailwind CSS Intellisense extension for VS Code.

Basic application structure

The following is an overview of what our project folder should look like:

link-previewer/
├─ functions/
│  ├─ generate-preview.js
│  └─ hello.js
├─ public/
│  ├─ favicon.ico
│  ├─ img/
│  │  └─ icons/
│  ├─ index.html
│  └─ robots.txt
├─ src/
│  ├─ main.js
│  ├─ App.vue
│  ├─ registerServiceWorker.js
│  ├─ assets/
│  │  ├─ css/
│  │  │  └─ main.css
│  │  └─ logo.png
│  ├─ components/
│  │  └─ LinkPreviewer.vue
│  ├─ router/
│  │  └─ index.js
│  ├─ store/
│  │  └─ index.js
│  └─ views/
│     ├─ About.vue
│     └─ Home.vue
├─ .git
├─ .gitignore
├─ .browserslistrc
├─ .eslintrc.js
├─ babel.config.js
├─ netlify.toml
├─ package-lock.json
├─ package.json
├─ postcss.config.js
├─ README.md
└─ tailwind.config.js

Introduction to Netlify function

Netlify Functions is a netlify product that simplifies the process of creating and deploying serverless functions. According to the home page of the product, it is used to:

Deploy server-side code that acts as an API endpoint, runs automatically in response to events, or processes more complex jobs in the background.

A basic Netlify function file exports a handler method using the following syntax:

exports.handler = async function(event, context){
  return {
    statusCode: 200,
    body: JSON.stringify({message: "Hello World!"})
  }
}

Netlify provides event and parameters when calling / calling functions. context when the endpoint of a function is called, the handler receives an event object like this:

{
  "path": "Path parameter (original URL encoding)",
  "httpMethod": "Incoming request's method name",
  "headers": {Incoming request headers},
  "queryStringParameters": {Query string parameters},
  "body": "A JSON string of the request payload",
  "isBase64Encoded": "A boolean flag to indicate if the applicable request payload is Base64-encoded"
}

On the other hand, the context parameter includes information about the context of the calling function.

In the function, we return an object with two important properties:

  • statusCode,200 in this case
  • body, this is a string object.

This function will / netlify/functions/hello is called from our site when successful, and it will return 200 status code and message "Hello, World!".

Now that we know how Netlify functions work, let's look at them in practice.

Create our first Netlify function

To create our first Netlify function, we will use functions / Hello JS create a new file in the project directory and enter the following:

// functions/hello.js
exports.handler = async function(event, context){
  return {
    statusCode: 200,
    body: JSON.stringify({message: "Hello World!"})
  }
}

Once we have created the function file, we must make some necessary configuration so that we can run our functions locally.

Set Netlify configuration

We will Netlify Toml creates a file in the root directory of the project folder and tells Netlify where to find our function:

# ./netlify.toml

[functions]
  directory = "./functions"

Netlify will now locate and deploy functions in the folder at build time.

Installing Netlify CLI

To run our functions locally without having to deploy to Netlify, we need to install Netlify CLI. CLI allows us to deploy projects with some excellent capabilities and Netlify functions locally.

To install the CLI, make sure you have node JS version 10 or later, and then run:

npm install netlify-cli -g

This will install the Netlify CLI globally, so we can run commands from any directory. To get the version, usage, etc., we can run:

netlify 

Running applications using Netlify Dev

To run our project locally using Netlify CLI, stop the development server (if it is active) and run:

netlify dev

This is what we should see:

If you look closely, you will find that something has happened there:

  • Netlify tried to bring us The environment variables in the env file are injected into the construction process, and then our netlify function can access these variables. In this case, we didn't Env file, so it loads process
  • Second, it loads or deploys Functions located in the Functions directory. The Functions server is deployed on a different random port - 36647
  • Finally, it will automatically detect what framework the application is built with and run the necessary build process to deploy the application. In this case, you can see "start Netlify Dev using Vue.js". It also supports React and other popular frameworks.

Netlify and then http://localhost:8888.

Now that our server has started and our function has been loaded, we can call it. By default, we can access our functions using the following route: / netlify/functions/<function name>.

One important thing to note is that we do not need to specify the port on which the Functions server runs. We can use the default route above to communicate with our Functions server. Netlify automatically parses URL s and ports in the background.

If we ask
http://localhost:8888/.netlify/functions/helloGET Send the request and we deserve it {"message":"Hello, World!"}

great! Our first server free feature works!

Create preview function API

Now that our Netlify function is working, we can start building the preview API. The following is a brief description of what our Functions API will do:

  • It receives the destination URL that will be sent from the front end
  • It passes data to the puppeter
  • Puppeter then launches a new headless browser instance
  • The puppeter opens a new page in the browser and navigates to the target URL
  • Then, the puppeter extracts the contents of < title > and < meta > tags for the description of the target page
  • It captures a screenshot of the page
  • It sends the screenshot data back to the front end

Now that we have a basic understanding of what the Functions API will do, we can start creating Functions. Let's start by installing and setting up puppeter for Netlify Functions.

Installing and configuring puppeter

Puppeter is a Node library that provides an advanced API to control headless Chrome or Chromium browsers. It can also be configured to use full (non headless) Chromium or Chromium. You can use puppeter to do most of the operations manually in the browser. More information about the puppeter can be found in the puppeter documentation.

In order to start using puppeter, we will install it into our project.

Puppet division for local development and production

Puppeter has downloaded the latest version of Chromium (about 170MB macOS, about 282MB Linux, about 280MB Windows), which is guaranteed to be used with API.

We cannot use the complete puppeter package for production. This is because the maximum size of Netlify Functions is 50MB, while the Chromium package is too large.

Thanks to this very useful article by Ire Aderinokun, we can still use puppeter in local and production environments. This is what we must do:

The puppeter is installed as a development dependency for local deployment:

npm i puppeteer --save-dev

In order for puppeter to work in local and production environments, we must install puppeter core and chrome AWS lambda as production dependencies.

Here you can see the difference between and puppeter. However, the main difference is that Chromium is not automatically downloaded during installation. puppeteer-core puppeteer-core

Since the puppeter core does not download the browser, we will install chrome AWS Lambda, a "Chromium binary for AWS Lambda and Google Cloud Functions", which we can use in the Netlify function. These are packages that can be used in production:

npm i puppeteer-core chrome-aws-lambda --save-prod

Now that we have installed our package, let's create our functions.

Use installed browser for puppeter

If puppeter installs a complete browser locally, it will become a problem, which may be due to slow network speed or bandwidth problems. One solution is to use the Chrome or Chromium browser we have installed to run puppeter.

What we need is the path of the browser on the local machine. We will use it as our executablePath, and we will pass it to the puppeter Launch() method. This tells puppeter where to find the browser executable.

If you don't know where to find the executable path, open your browser and go to chrome://version/ To display the version of chrome.

. env copy the path and create a file in the root directory of the project.

# ./.env
EXCECUTABLE_PATH=<path to chrome>

To get According to the contents of env file, we will install another package - dotenv:

npm install dotenv

Now that we have successfully installed the package, let's create the Netlify function.

Create generation preview function

Create a new file,
./functions/generate-preview.js:

// ./functions/generate-preview.js

const chromium = require('chrome-aws-lambda')
const puppeteer = require('puppeteer-core')

exports.handler = async function (event, context) {
  // parse body of POST request to valid object and
  // use object destructuring to obtain target url
  const { targetURL } = JSON.parse(event.body)

  // launch browser
  const browser = await puppeteer.launch({
    args: chromium.args,
    // get path to browser
    executablePath: process.env.EXCECUTABLE_PATH || await chromium.executablePath,
    headless: true
  })

  // open new page in browser
  const page = await browser.newPage()

  // set the viewport of the page
  await page.setViewport({
    width: 768,
    height: 425,
    deviceScaleFactor: 1
  })

  // set the prefers-color-scheme to dark
  await page.emulateMediaFeatures([
    {name: 'prefers-color-scheme', value:'dark'}
  ])

  // navigate to target URL and get page details and screenshot
  try{
    ...
  }
}

In the above code, we did a lot of things. First, we get the event from the request payload in the targetURL body. This will be sent with the POST request.

Next, we use the chrome AWS lambda package to launch the browser. We use puppeter Do this by launching. This method accepts an object as a parameter with some optional properties. An important property we pass to this method is executablePath

We will assign executablePath to
process.env.EXCECUTABLE_PATH || await chromium.executablePath enables the package to find an available headless browser to start.

browser. After the newpage () browser starts, we use this method to open a new page in the browser. page.setViewport() we also use this method to set the required browser viewport for the page.

Note that we use keywords when await runs any function. This is because the puppeter works asynchronously, and some functions may take some time to execute.

We can also use
Puppeteerpage. The emulatemediafeatures () method defines the media features of the page, which adopts the array of media feature objects. This is how we set preferences color schemeto.

Get site metadata and screenshots

Next, we will navigate to the target URL and get our title, description and screenshot:

// ./functions/generate-preview.js

...
// navigate to target URL and get page details and screenshot
try {
  // navigate to the targetURL
  await page.goto(targetURL)

  // get the title from the newly loaded page
  const title = (await page.$eval(`head > title`, el => el.textContent) || null)

  // get the descriptions of the page using their CSS selectors
  const descriptions = await page.evaluate(() => {
    let descriptions = {}

    let desc = document.querySelector(`meta[name='description']`)
    let og = document.querySelector(`meta[property='og:description']`)
    let twitter = document.querySelector(`meta[property='twitter:description']`)

    desc ? descriptions.desc = desc.content : descriptions.desc = null
    og ? descriptions.og = og.content: descriptions.og = null
    twitter ? descriptions.twitter = twitter.content : descriptions.twitter = null

    return descriptions
  })

  // screenshot the page as a jpeg with a base64 encoding
  const screenshot = await page.screenshot({
    type: 'jpeg',
    encoding: 'base64'
  })

  // close the browser
  await browser.close()

  // send the page details 
  return {
    statusCode: 200,
    body: JSON.stringify({
      title,
      screenshot,
      descriptions
    })
  }

} catch (error) {

  // if any error occurs, close the browser instance 
  // and send an error code
  await browser.close()
  return {
    statusCode: 400,
    body: JSON.stringify({
      error
    })
  }
}

In the above code, we use a trycatch block to wrap our code, so that if there is any problem, we can start from await page Goto (target URL) navigates to the beginning of the target URL, and we can catch the error and send it to our front end. Providing an invalid URL may cause an error.

If the URL is valid, we use the method to get the title page$ Eval (), which is similar to
document. Queryselector is a common method in JavaScript. We pass in the CSS selector of the head > title title tag -- as the first parameter. We also passed a function El = > el Textcontent as the second parameter, where El is the parameter we passed to the function and is the title element. We can now use title textContent.

Note that all of these are enclosed in parentheses (()), and we have a | nullafter page$ eval. Title if page$ Eval() cannot get the page title, assign it null.

To get the description of the page, we will use this page Evaluate () method, which allows us to run some client-side JavaScript and return a value - descriptions to the allocated variable.

We pass a function as a parameter to the page Evaluate() method. In our document Queryselector is used to obtain various meta descriptions, such as < meta name = "description" content = "< site description >" / > Default description and < meta property = "og: description" content = "< site description >" / > opengraph description.

After obtaining the element, if the element exists or does not exist, we use the ternary operator to obtain the content and add it to the object. descriptionsnull

After getting the description, we use page The screenshot () method intercepts the page and closes the browser with close().

Finally, we send the page details in the body attribute to a JSON object, where the statusCode contains 200 If an error occurs in any of the previous steps, it will be captured in the catch block, and we will send a statusCodeof400 and an error message.

Test and deploy functions

Let's use the API tester to test our functionality. You can install the Postman or Talend API tester in the browser, or use the Thunder Client extension, which is the API tester of VS Code.

You can also use cURL:

curl -X POST -H "Content-Type: application/json" -d '{"paramName": "value"}' <URL>

netlify dev uses the command to run the function.

We can use the port of the function server or the default port of the 8888Netlify development server to send requests to our functions. I will use objects that contain the following objects
http://localhost:8888/.netlify/functions/generate-preview send POST request: targetURLbody

{
  "targetURL" : "https://miracleio.me"
}

When we send a request, this is the response we get.

We get a JSON object containing preview data:

{
  "title": "Miracleio | PortfolioX",
  "screenshot": "/9j/4AAQSkZJRgABAQAAAQABAAD...",
  "descriptions": {
    "desc": "Designer & Frontend Developer portfolio site. Built by Miracleio with love ❤",
    "og": "Designer & Frontend Developer portfolio site. Built by Miracleio with love ❤",
    "twitter": null
  }
}

Now that our serverless feature is working, let's see how to use it on the front end.

Build link preview function on the client

In order to interact with our generate preview function, we need to send a POST containing our targetURL

We will create a LinkPreview component that displays normal links. These components will pass their target URL as props. Before the component is installed into the application, it will send a POST request to our serverless function to get the preview data and display it when we hover over the link. targetURL

Create link preview component

First, let's create our link preview component
src/components/LinkPreviewer.vue.

In our < script >, we will get the link preview data by sending a request to our serverless function and save the data in the previewData object. We will use it later in the template to display the data:

// ./src/components/LinkPreviewer.vue
...

<script>
import { computed, onBeforeMount, ref } from '@vue/runtime-core'
  export default {
    // define targetURL as a prop
    props: ['targetURL'],
    setup(props) {
      // create a reactive previewData object using ref
      const previewData = ref({})

      // function to send a POST request containing the targetURL 
      // to the serverless function
      const generatePreview = async () => {
        try {
          const res = await fetch('/.netlify/functions/generate-preview', {
            method: 'POST',
            body: JSON.stringify({
              targetURL : props.targetURL
            })
          })

          const data = await res.json()
          return data
        } catch (err) {
          console.log(err)
          return null
        }
      }

      // run function before component is mounted
      onBeforeMount(async ()=>{
        // run generatePreview() to get the preview data and assign to previewData
        previewData.value = await generatePreview()

        // use object destructuring to get the different descriptions 
        // from the preview data
        const {desc, og, twitter} = previewData.value.descriptions

        // assign only one valid value to the description property 
        // in the previewData object
        previewData.value.description = computed(()=>(desc || og || twitter || ""))
      })

      // make the following entities available to the component
      return { generatePreview, previewData}
    }
  }
</script>

In the above code, we pass the targetURL to our component as props.

In setup(), we pass props as a parameter so that we can access things like targetURL

Then we use: to create a responsive peviewData object. In a new function, we use it to send an inclusive request to our serverless function. This function returns a response or if an error occurs. refconst previewData = ref({})generatePreview()fetchPOSTtargetURLnull

Next, to run the function before mounting the component, we use the onBeforeMount() hook. We pass an async function as an argument. Within the function, we assign previewdata Value to the generatePreview() function. desc, og, twitter, and then get the description from the descriptions attribute ().

In order to get the description that will be displayed in the preview, we will assign
previewData.value.description (description | og | twitter | "") In this way, the first attribute with value is assigned to description

Do this to display preview data in our template:

<!-- ./src/components/LinkPreviewer.vue -->

<template>
  <div class="inline relative">
    <!-- display targetURL link -->
    <a class="link underline text-blue-600" 
       :href="targetURL" 
       :target="previewData ? previewData.title : '_blank'">
       {{targetURL}} 
    </a>

    <!-- display preview data if object exists -->
    <div v-if="previewData" class="result-preview absolute top-8 left-0 w-72 transform translate-y-4 opacity-0 invisible transition bg-white overflow-hidden rounded-md shadow-lg z-10">

      <!-- display image using the base64 screenshot data -->
      <img v-if="previewData.screenshot"
           :src="`data:image/jpeg;base64,${previewData.screenshot}`"
           :alt="previewData.description" />

      <!-- display title and description -->
      <div class="details p-4 text-left">
        <h1 class=" font-extrabold text-xl"> {{previewData.title}} </h1>
        <p> {{previewData.description}} </p>
      </div>
    </div>
  </div>
</template>

<script> ... </script>

<style scoped>
  .link:hover ~ .result-preview{
    @apply visible opacity-100 translate-y-0;
  }
</style>

In the above code, in order to display our image (essentially a base64 string), we must pass the string together with data such as image type and encoding to the src - "attribute.

This is our preview.link Vue components. Let's see its actual effect. Yes/ src/views/Home.vue:

<!-- ./src/views/Home.vue -->

<template>
  <main class="home">
    <header>
      <h1>Welcome to the link previewer app!</h1>
      <p>Here are some links that you can preview by hovering on them</p>
    </header>
    <ul class=" mb-4">
      <!-- render LinkPreviewer component for each demolink -->
      <li v-for="link in demoLinks" :key="link">
        <link-previewer :targetURL="link" />
      </li>
    </ul>
    <!-- input field to add new links -->
    <input class=" p-2 ring ring-blue-600 rounded-lg shadow-md" type="url" @keyup.enter="addLink" required placeholder="enter valid url">
  </main>
</template>

<script>
import { ref } from '@vue/reactivity'
import LinkPreviewer from '../components/LinkPreviewer.vue'

export default{
  components: { LinkPreviewer },
  setup(){
    // demo links
    const demoLinks = ref([
      'http://localhost:5000',
      'https://google.com',
      'https://miracleio.me',
      'https://miguelpiedrafita.com/'
    ])

    // function to add new links to the demoLinks array
    const addLink = ({target}) => {
      demoLinks.value.push(target.value)
      target.value = ""
    }

    return {demoLinks, addLink}
  }
}
</script>

In our home In the Vue file, we basically use the demoLinks link array to present the LinkViewer component list, which we pass to the props of the targetURL component.

We also have an < input > element that we use to dynamically add more LinkViewer components to the list.

This is what our simple application looks like now.

Sweet! Our application is valid. Since we have been using the Netlify CLI to run locally, let's see how to deploy to Netlify using the CLI.

Deploy application to Netlify

Before we deploy our application to Netlify, we must build our application for production:

npm run build

This will build our application and create a dist / folder where we can deploy to the production environment.

Next, we need to log in to our Netlify account:

netlify deploy

This will allow you to log in to your Netlify account in a browser.

After we authorize the application, we can link our project to a new site. Netlify will ask us a lot of questions:

  • What are you up to? Select Create and configure a new site.
  • Team? Select < your team >.
  • Choose a unique website name? Select < site name >.
  • Please provide the publication directory (e.g. "public" or "dist" or "."). Enter dist.

After that, Netlify will upload our files and deploy them to our new site.

Deploy using GitHub

Alternatively, we can decide to deploy our website from GitHub. All you have to do is log in to GitHub, create a new repository, and copy the URL to our newly created repository.

Then we run the following command in the project folder:

git init
git add .
git commit -m "first commit"
git branch -M main
git remote add origin https://github.com/miracleonyenma/link-previewer.git
git push -u origin main

Note: due to authentication problems, you may not be able to push from the terminal to your repository, and you may receive the following message from Git: "support for password authentication has been removed on August 13, 2021. Please use a personal access token." This means that you must create a personal access token (PAT) and use it to log in. To do this, go to GitHub token settings and generate a new token. Select all required permissions. Make sure you have access to the repository. After generating the pat, copy it and save it somewhere. Then Git push - U origin maintry the command again and paste your pat when asked for your password.

After pushing the project to GitHub, go to Netlify to create a new site from GitHub.

Follow the steps to select the repository and enter the build settings for the project. For our Vue project, the build command is npm run build and the deployment directory is dist

After that, click Deploy site.

Netlify will deploy the site, and we can preview our site by clicking the deployment link provided. We can view our functions by going to functions from the top menu.

You can select a feature to view more details and logs.

conclusion

We have been able to use the Netlify function to create and deploy serverless functions through Netlify. We also saw how to interact with Vue front-end functions. This time, we are used to taking screenshots and getting data from other websites, and use it to build a link previewer component, but there is still a lot we can do. With the server free function, we can do more things on the front end without bothering to set up the back-end server.

If this article is helpful to you, don't forget to give me a 3-company, like, forward, comment,, I'll see you next time.

Collecting is equal to whoring in vain. Praise is the truth.

Dear friends, if you need JAVA interview documents, please like + forward them. After paying attention to me, I can get free materials by private letter 333

Topics: Java Front-end Spring Vue.js serverless