Android ARCore – Distance from Camera

ARCore Basics

Augmented Reality basically allows us to show virtual objects in a real-world environment and allow interactions with them.

ARCore requires :

• Android N+
• OpenGL 3.0 +

Knowledge in OpenGL 3d graphics programming is not necessary since Sceneform SDK is introduced for the same.

Sceneform allows us to create 3d models. It consists of ShapeFactory and MaterialFactory classes that allow us to create renderable objects from simple shapes and materials.
Shapes like a sphere, cube, cylinder can easily we created as we shall see later in this tutorial.

ARFragment is used to create an Ar scene and AR Session (both are created automatically).
Using an ARFragment we detect planes or feature points.

setOnTapArPlaneListener is set on the ArFragment to listen to changes whenever the click event takes place.

Few terms that form of the core of ARCore are listed below with descriptions:

• Scene – This is where our 3D objects are rendered.
• HitResult – On tap, this provides us the location of a real-world object. Basically, it estimates the position by finding the first point of intersection with an imaginary ray of light coming from infinity.
• Anchor – A fixed location in real-world in terms of x,y,z coordinates in 3D space. Just like a ship anchor
• Pose – provides the position and orientation of the object in the scene.
• AnchorNode – This is the node that automatically positions itself in the world. This is the first node that gets set when a plane is detected.
• TransformableNode – This node is where we set our 3d object. It can interact, scale, transform, rotate according to the user interactions.

In the next section, we’ll be developing our AR application in which we’ll calculate the distance of an object from us i.e the camera.

How is this done?
Once we place an object on a plane, we know it’s a pose.
As our scene changes, the frame gets updated. From the frame, we can get the camera pose.
Now that we have the two poses, the distance between them is just a Math formula!

Project Structure

Android Arcore Project Structure

Add the following dependency to the build.gradle:

implementation 'com.google.ar.sceneform.ux:sceneform-ux:1.10.0'

In the AndroidManifest file we need to add the following:

    <uses-permission android:name="android.permission.CAMERA" />
    <uses-feature
        android:name="android.hardware.camera.ar"
        android:required="true" />

Inside the application tag add the following:

<meta-data
                android:name="com.google.ar.core"
                android:value="required" />

Code

The code for the activity_main.xml is given below:

<?xml version="1.0" encoding="utf-8"?>
<FrameLayout xmlns:android="https://schemas.android.com/apk/res/android"
    android:layout_width="match_parent"
    android:layout_height="match_parent">

    <fragment
        android:id="@+id/ux_fragment"
        android:name="com.google.ar.sceneform.ux.ArFragment"
        android:layout_width="match_parent"
        android:layout_height="match_parent" />

    <TextView
        android:id="@+id/tvDistance"
        android:layout_width="match_parent"
        android:layout_height="wrap_content"
        android:layout_gravity="bottom"
        android:background="@android:color/black"
        android:gravity="center"
        android:padding="8dp"
        android:text="Distance from camera"
        android:textColor="@android:color/white"
        android:textSize="20sp" />

</FrameLayout>

The code for the MainActivity.java is given below:

package com.journaldev.androidarcoredistancecamera;

import androidx.appcompat.app.AppCompatActivity;

import android.app.Activity;
import android.app.ActivityManager;
import android.content.Context;
import android.os.Bundle;
import android.util.Log;
import android.widget.TextView;
import android.widget.Toast;

import com.google.ar.core.Anchor;
import com.google.ar.core.Frame;
import com.google.ar.core.Pose;
import com.google.ar.sceneform.AnchorNode;
import com.google.ar.sceneform.FrameTime;
import com.google.ar.sceneform.Scene;
import com.google.ar.sceneform.math.Vector3;
import com.google.ar.sceneform.rendering.Color;
import com.google.ar.sceneform.rendering.MaterialFactory;
import com.google.ar.sceneform.rendering.ModelRenderable;
import com.google.ar.sceneform.rendering.ShapeFactory;
import com.google.ar.sceneform.ux.ArFragment;
import com.google.ar.sceneform.ux.TransformableNode;

import java.util.Objects;

public class MainActivity extends AppCompatActivity implements Scene.OnUpdateListener {


    private static final double MIN_OPENGL_VERSION = 3.0;
    private static final String TAG = MainActivity.class.getSimpleName();

    private ArFragment arFragment;
    private AnchorNode currentAnchorNode;
    private TextView tvDistance;
    ModelRenderable cubeRenderable;
    private Anchor currentAnchor = null;

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);

        if (!checkIsSupportedDeviceOrFinish(this)) {
            Toast.makeText(getApplicationContext(), "Device not supported", Toast.LENGTH_LONG).show();
        }

        setContentView(R.layout.activity_main);

        arFragment = (ArFragment) getSupportFragmentManager().findFragmentById(R.id.ux_fragment);
        tvDistance = findViewById(R.id.tvDistance);


        initModel();

        arFragment.setOnTapArPlaneListener((hitResult, plane, motionEvent) -> {
            if (cubeRenderable == null)
                return;

            // Creating Anchor.
            Anchor anchor = hitResult.createAnchor();
            AnchorNode anchorNode = new AnchorNode(anchor);
            anchorNode.setParent(arFragment.getArSceneView().getScene());

            clearAnchor();

            currentAnchor = anchor;
            currentAnchorNode = anchorNode;


            TransformableNode node = new TransformableNode(arFragment.getTransformationSystem());
            node.setRenderable(cubeRenderable);
            node.setParent(anchorNode);
            arFragment.getArSceneView().getScene().addOnUpdateListener(this);
            arFragment.getArSceneView().getScene().addChild(anchorNode);
            node.select();


        });


    }

    public boolean checkIsSupportedDeviceOrFinish(final Activity activity) {

        String openGlVersionString =
                ((ActivityManager) Objects.requireNonNull(activity.getSystemService(Context.ACTIVITY_SERVICE)))
                        .getDeviceConfigurationInfo()
                        .getGlEsVersion();
        if (Double.parseDouble(openGlVersionString) < MIN_OPENGL_VERSION) {
            Log.e(TAG, "Sceneform requires OpenGL ES 3.0 later");
            Toast.makeText(activity, "Sceneform requires OpenGL ES 3.0 or later", Toast.LENGTH_LONG)
                    .show();
            activity.finish();
            return false;
        }
        return true;
    }

    private void initModel() {
        MaterialFactory.makeTransparentWithColor(this, new Color(android.graphics.Color.RED))
                .thenAccept(
                        material -> {
                            Vector3 vector3 = new Vector3(0.05f, 0.01f, 0.01f);
                            cubeRenderable = ShapeFactory.makeCube(vector3, Vector3.zero(), material);
                            cubeRenderable.setShadowCaster(false);
                            cubeRenderable.setShadowReceiver(false);
                        });
    }

    private void clearAnchor() {
        currentAnchor = null;


        if (currentAnchorNode != null) {
            arFragment.getArSceneView().getScene().removeChild(currentAnchorNode);
            currentAnchorNode.getAnchor().detach();
            currentAnchorNode.setParent(null);
            currentAnchorNode = null;
        }
    }

    @Override
    public void onUpdate(FrameTime frameTime) {
        Frame frame = arFragment.getArSceneView().getArFrame();

        Log.d("API123", "onUpdateframe... current anchor node " + (currentAnchorNode == null));


        if (currentAnchorNode != null) {
            Pose objectPose = currentAnchor.getPose();
            Pose cameraPose = frame.getCamera().getPose();

            float dx = objectPose.tx() - cameraPose.tx();
            float dy = objectPose.ty() - cameraPose.ty();
            float dz = objectPose.tz() - cameraPose.tz();

            ///Compute the straight-line distance.
            float distanceMeters = (float) Math.sqrt(dx * dx + dy * dy + dz * dz);
            tvDistance.setText("Distance from camera: " + distanceMeters + " metres");


            /*float[] distance_vector = currentAnchor.getPose().inverse()
                    .compose(cameraPose).getTranslation();
            float totalDistanceSquared = 0;
            for (int i = 0; i < 3; ++i)
                totalDistanceSquared += distance_vector[i] * distance_vector[i];*/
        }
    }
}

In the above code we do the following things:

• Check if the phone is AR compatible.
• Create a 3d cube shaped model.
• Add it on tap once the plane is detected.
• Update the distance from the camera to the anchor in every frame.
• Once you tap again, clear the previous anchor.

We can set a tap listener on the transformable node as well node.setOnTapListener{} and do stuff with the node when it is clicked ( play sound etc.).

The output of the application in action is given below:

Android Arcore Distance Camera Output

Andorid Clipboard – Copy Paste

You must have noticed that EditText has a built in Clipboard Manager for Copy Paste functionality.
But a TextView doesn’t. So in order to allow Copy Pasting TextViews, we need to register the ContextMenu.

Hence in order to receive the menu events, we need to registerForContextMenu.
Once this is done, you can long-press on the TextView to display the menu.

But where’s the menu?

The menu is created in the onCreateContextMenu() method.
The menu items actions are set in the onContextItemSelected method.

Copying Text To Clipboard

It’s easy to copy TextView text onto the Clipboard. You just need to set the ClipData type as newPlainText and pass the string.

Example:

ClipboardManager manager = (ClipboardManager) getSystemService(CLIPBOARD_SERVICE); 
ClipData clipData = ClipData.newPlainText("text", textView.getText());
manager.setPrimaryClip(clipData);

Pasting Text To Clipboard

Now that you have copied the text onto the Clipboard, just paste it using the following piece of code:

ClipData pasteData = manager.getPrimaryClip();
ClipData.Item item = pasteData.getItemAt(0);
String paste = item.getText().toString();

Here we’re pasting the first data from the Clipboard. If we have multiple copied texts, they can be pasted using pasteData.getItemAt(1) and so on.

Now let’s jump onto the implementation part of this article.
In the next section, we’ll be developing a simple Android Application in which you can copy-paste text from one TextView to another.

Project Structure

Android Textview Copy Paste Project Structure

Code

The code for the activity_main.xml layout is given below:

<?xml version="1.0" encoding="utf-8"?>
<androidx.constraintlayout.widget.ConstraintLayout xmlns:android="https://schemas.android.com/apk/res/android"
    xmlns:app="https://schemas.android.com/apk/res-auto"
    xmlns:tools="https://schemas.android.com/tools"
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    tools:context=".MainActivity">

    <TextView
        android:id="@+id/tvCopy"
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:text="Welcome To Journaldev.com"
        android:padding="8dp"
        app:layout_constraintBottom_toBottomOf="parent"
        app:layout_constraintLeft_toLeftOf="parent"
        app:layout_constraintRight_toRightOf="parent"
        app:layout_constraintTop_toTopOf="parent" />


    <TextView
        android:id="@+id/tvPaste"
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:text="Paste here"
        android:padding="8dp"
        app:layout_constraintBottom_toBottomOf="parent"
        app:layout_constraintLeft_toLeftOf="parent"
        app:layout_constraintRight_toRightOf="parent"
        app:layout_constraintTop_toBottomOf="@+id/tvCopy" />

</androidx.constraintlayout.widget.ConstraintLayout>

The code for the MainActivity.java class is given below:

package com.journaldev.androidtextviewcopypaste;

import androidx.annotation.NonNull;
import androidx.appcompat.app.AppCompatActivity;

import android.content.ClipData;
import android.content.ClipboardManager;
import android.os.Bundle;
import android.view.ContextMenu;
import android.view.MenuItem;
import android.view.View;
import android.widget.TextView;
import android.widget.Toast;

public class MainActivity extends AppCompatActivity {


    TextView tvCopy, tvPaste;

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_main);

        tvCopy = findViewById(R.id.tvCopy);
        tvPaste = findViewById(R.id.tvPaste);

        registerForContextMenu(tvCopy);
        registerForContextMenu(tvPaste);
    }

    @Override
    public void onCreateContextMenu(ContextMenu menu, View v, ContextMenu.ContextMenuInfo menuInfo) {
        super.onCreateContextMenu(menu, v, menuInfo);

        menu.setHeaderTitle("Options");
        switch (v.getId()) {
            case R.id.tvCopy:
                menu.add(0, v.getId(), 0, "Copy");
                TextView textView = (TextView) v;
                ClipboardManager manager = (ClipboardManager) getSystemService(CLIPBOARD_SERVICE);
                ClipData clipData = ClipData.newPlainText("text", textView.getText());
                if (manager != null) {
                    manager.setPrimaryClip(clipData);
                }

                break;
            case R.id.tvPaste:
                menu.add(0, v.getId(), 0, "Paste");
                break;
        }
    }

    @Override
    public boolean onContextItemSelected(@NonNull MenuItem item) {

        switch (item.getItemId()) {
            case R.id.tvCopy:
                Toast.makeText(getApplicationContext(), "Copy Clicked", Toast.LENGTH_LONG).show();
                break;
            case R.id.tvPaste:
                ClipboardManager manager = (ClipboardManager) getSystemService(CLIPBOARD_SERVICE);
                if (manager != null && manager.getPrimaryClip() != null && manager.getPrimaryClip().getItemCount() > 0) {
                    tvPaste.setText(manager.getPrimaryClip().getItemAt(0).getText().toString());
                }

                break;
        }


        return super.onContextItemSelected(item);


    }
}

getItemCount() is used to get the items present in the clipboard.
You can also set the clipboard data to be copied in the onContextItemSelected copy menu item action.

Output

The output of the above application in action is given below:

Android Textview Copy Paste Output

Android — Integrating Fingerprint authentication in your apps

Fingerprint authentication has been a huge leap towards a better user experience in smartphones. Having this golden feature in your user’s phone and trying to integrate this feature in your applications to increase security and the bonus of better user experience is worth a shot. Let’s see how we can quickly integrate a simple FingerPrint authentication into an android app.

While integrating Fingerprint auth in our applications, we need to take care of quite a lot of things concerning the hardware of the device our users are using. We know that only devices running android Marshmallow and above have support for Fingerprint scanners. Additionally to use the Android’s built-in Fingerprint features the user needs to have enabled fingerprint security in the device and have at least one fingerprint registered in the device. Let’s keep these points in mind and we need to check these things before dispatching a listener for fingerprint authentication.
So, Let’s get started!

Permission for Fingerprint use

We need our application’s manifest file to describe that our application needs permission to use Fingerprint. Let’s add the following line inside the manifest tag inside the AndroidManifest.xml file.

<uses-permission android:name="android.permission.USE_FINGERPRINT" />

Checking Fingerprint availability

As I mentioned above, we need to verify that the user runs appropriate Android version, the device has the necessary hardware for the Fingerprint scanner and the user has at least one fingerprint already registered in the device.
Write down the following method in your activity’s class or create a separate java file and drop this method in there.

Creating a Fingerprint Helper class

Next, we need to create a helper class by extending the FingerprintManager.AuthenticationCallback and we'll create an interface inside this class for the authentication listner.

@RequiresApi(api = Build.VERSION_CODES.M)
public class FingerprintHelper extends AuthenticationCallback{

    private FingerprintHelperListener listener;

    public FingerprintHelper(FingerprintHelperListener listener) {
        this.listener = listener;
    }

    //interface for the listner
    interface FingerprintHelperListener {
        void authenticationFailed(String error);
        void authenticationSuccess(FingerprintManager.AuthenticationResult result);
    }

}

Additionally, we need to create methods to start authentication, cancel the listner and implement the methods in the AuthenticationCallback to interact with our listner. Now our FilgerprintHelper will look like this.

Implementing the listener in Activity

Once we have created the FingerprintHelper.java and the FingerprintHelperListener inside it, we can implement the listner in our Activity.

public class LoginActivity extends AppCompatActivity implements FingerprintHelper.FingerprintHelperListener {

   //Your onCreate and other pieces of code goes here
   ...
   ...
   //You class should implement the listener and override the below two methods
    @Override
    public void authenticationFailed(String error) {
        //Authentication failed. ie user tried an invalid fingerprint
        Toast.makeText(this, "Invalid Fingeprint", Toast.LENGTH_LONG).show();
    }

    @Override
    public void authenticationSuccess(FingerprintManager.AuthenticationResult result) {
        //Yaay! we have authenticated the user using Fingerprint.
        ...
        //add your logic to continue here.
    }
}

We need to create variables of FingerprintManager and FingeprintHelper in our activity.

private FingerprintHelper fingerprintHelper;
private FingerprintManager fingerprintManager;

Next, we’ll check for the Fingerprint settings using the function checkFingerprintSettings we discussed above in this post and start listneing when the Activity starts and cancel the listening when the activity is paused ie. app went to background. So add the following line of code inside your activity by overriding the onResume() and the onPause() method.

@RequiresApi(api = Build.VERSION_CODES.M)
@Override
protected void onResume() {
    super.onResume();
     //Check for the fingerprint permission and listen for fingerprint
     //add additional checks along with this condition based on your logic
    if (checkFingerPrintSettings(this)) {
        //Fingerprint is available, update the UI to ask user for Fingerprint auth
        //start listening for Fingerprint
        fingerprintHelper = new FingerprintHelper(this);
        fingerprintManager = (FingerprintManager) getSystemService(FINGERPRINT_SERVICE);
        fingerprintHelper.startAuth(fingerprintManager, null);
     } else {
        Log.d(TAG, "Finger print disabled due to No Login credentials or no Fingerprint");
     }
}

@RequiresApi(api = Build.VERSION_CODES.M)
@Override
protected void onPause() {
   super.onPause();
   if (fingerprintHelper != null)
       fingerprintHelper.cancel();
}

That’s it! You’re ready to roll with the new feature on your app. Now your Activity class should look something like this.