firebase2graphql: improve readme structure, update git and add more tests (#666)

This commit is contained in:
Rishichandra Wawhal 2018-10-08 11:42:42 +05:30 committed by Shahidh K Muhammed
parent 8454e70614
commit fbd220621d
8 changed files with 259 additions and 31 deletions

View File

@ -8,6 +8,25 @@ A CLI tool to help you try realtime GraphQL on your firebase data. It takes data
![GIF](https://graphql-engine-cdn.hasura.io/assets/firebase2graphql/demo.gif) ![GIF](https://graphql-engine-cdn.hasura.io/assets/firebase2graphql/demo.gif)
## Contents
- [Getting started](#quick-start)
- [Installation](#installation)
- [Usage](#usage)
- [Command](#command)
- [Usage comparison](#usage-comparison---firebase-sdk-vs-graphql)
- [Authentication](#authentication)
- [Next steps](#next-steps)
- [More information](#more-information)
- [Working](#working)
- [Normalization](#normalization)
- [Automatic](#automatic)
- [Manual](#manual)
- [Duplicates](#duplicates)
- [Overwrite](#overwrite)
- [Feedback](#feedback)
## Quick start ## Quick start
1. Quickly get the GraphQL Engine running by clicking this button: 1. Quickly get the GraphQL Engine running by clicking this button:
@ -177,8 +196,6 @@ Check out [next steps](#next-steps).
## Installation ## Installation
### CLI
```bash ```bash
npm install -g firebase2graphql npm install -g firebase2graphql
``` ```
@ -215,22 +232,7 @@ firebase2graphql URL [flags]
- `-v --version`: show CLI version - `-v --version`: show CLI version
- `-h, --help`: show CLI help - `-h, --help`: show CLI help
## Next steps ## Usage comparison - Firebase SDK vs GraphQL
Once you have imported your data, it is recommended that you make it production ready.
1. Normalize the data by [removing duplicates](#duplicates).
2. Explore the GraphQL Engine Console to play with things such as
- [Relationships](https://docs.hasura.io/1.0/graphql/manual/schema/relationships/index.html)
- [Permissions](https://docs.hasura.io/1.0/graphql/manual/auth/index.html)
- Using SQL
- [Set up async business logic using event triggers](https://docs.hasura.io/1.0/graphql/manual/event-triggers/index.html)
- [Create new tables](https://docs.hasura.io/1.0/graphql/manual/schema/basics.html)
3. Set appropriate permissions. GraphQL Engine comes with [fine grained control layer](https://docs.hasura.io/1.0/graphql/manual/auth/index.html) that can be integrated with any standard Auth provider.
## Usage Comparison - Firebase SDK vs GraphQL
A typical query to do a single read from the database using [Firebase SDK](https://firebase.google.com/docs/reference/), (javascript) would look something like: A typical query to do a single read from the database using [Firebase SDK](https://firebase.google.com/docs/reference/), (javascript) would look something like:
@ -275,7 +277,26 @@ mutation {
} }
``` ```
## Things to know about implementation ## Authentication
Hasura can be integrated with most standard Authentication mechanisms including Firebase and Auth0. [Check out the authentication docs here](https://docs.hasura.io/1.0/graphql/manual/auth/index.html).
## Next steps
Once you have imported your data, it is recommended that you make it production ready.
1. [Normalize the data](#normalization)
2. Explore the GraphQL Engine Console to play with things such as
- [Relationships](https://docs.hasura.io/1.0/graphql/manual/schema/relationships/index.html)
- [Permissions](https://docs.hasura.io/1.0/graphql/manual/auth/index.html)
- Using SQL
- [Set up async business logic using event triggers](https://docs.hasura.io/1.0/graphql/manual/event-triggers/index.html)
- [Create new tables](https://docs.hasura.io/1.0/graphql/manual/schema/basics.html)
3. Set appropriate permissions. GraphQL Engine comes with [fine grained control layer](https://docs.hasura.io/1.0/graphql/manual/auth/index.html) that can be integrated with any standard Auth provider.
## More information
### Working ### Working
@ -287,6 +308,8 @@ If you use the flag `--normalize`, the CLI finds out if the children tables are
### Normalization ### Normalization
#### Automatic
The CLI provides a flag called `--normalize` if you want to normalize your denormalized database. The CLI provides a flag called `--normalize` if you want to normalize your denormalized database.
A lot of guess-work is done by the CLI while normalizing the database. Here are some thing you need to know: A lot of guess-work is done by the CLI while normalizing the database. Here are some thing you need to know:
@ -295,6 +318,143 @@ A lot of guess-work is done by the CLI while normalizing the database. Here are
2. Children tables are deleted if they are detected to be duplicates of some other root or child table. 2. Children tables are deleted if they are detected to be duplicates of some other root or child table.
3. In case of some children tables, when the data lacks a unique identifier, an extra unique field is added. In most cases, this field gets deleted while mergine a duplicate table with the original table. 3. In case of some children tables, when the data lacks a unique identifier, an extra unique field is added. In most cases, this field gets deleted while mergine a duplicate table with the original table.
#### Manual
In some cases, due to inconsistent field names or due to insufficient data, the CLI might not be able to normalize your database; but it makes sure that you do not lose any data.
In such cases, you can normalize the data yourself. Lets look at an example.
Consider this firebase database. This is the database of the official Firebase example app:
```json
{
"posts" : {
"-LMbLFOAW2q6GO1bD-5g" : {
"author" : "Eena",
"authorPic" : "https://lh4.googleusercontent.com/-vPOIBOxCUpo/AAAAAAAAAAI/AAAAAAAAAFo/SKk9hpOB7v4/photo.jpg",
"body" : "My first post content\nAnd body\nANd structure",
"starCount" : 0,
"title" : "My first post",
"uid" : "4UPmbcaqZKT2NdAAqBahXj4tHYN2"
},
"-LMbLIv6VKHYul7p_PZ-" : {
"author" : "Eena",
"authorPic" : "https://lh4.googleusercontent.com/-vPOIBOxCUpo/AAAAAAAAAAI/AAAAAAAAAFo/SKk9hpOB7v4/photo.jpg",
"body" : "AKsdjak\naklsdjaskldjklas\nasdklfjaklsdfjklsda\nasdklfjasklf",
"starCount" : 0,
"title" : "Whatta proaaa",
"uid" : "4UPmbcaqZKT2NdAAqBahXj4tHYN2"
}
},
"user-posts" : {
"4UPmbcaqZKT2NdAAqBahXj4tHYN2" : {
"-LMbLFOAW2q6GO1bD-5g" : {
"author" : "Eena",
"authorPic" : "https://lh4.googleusercontent.com/-vPOIBOxCUpo/AAAAAAAAAAI/AAAAAAAAAFo/SKk9hpOB7v4/photo.jpg",
"body" : "My first post content\nAnd body\nANd structure",
"starCount" : 0,
"title" : "My first post",
"uid" : "4UPmbcaqZKT2NdAAqBahXj4tHYN2"
},
"-LMbLIv6VKHYul7p_PZ-" : {
"author" : "Eena",
"authorPic" : "https://lh4.googleusercontent.com/-vPOIBOxCUpo/AAAAAAAAAAI/AAAAAAAAAFo/SKk9hpOB7v4/photo.jpg",
"body" : "AKsdjak\naklsdjaskldjklas\nasdklfjaklsdfjklsda\nasdklfjasklf",
"starCount" : 0,
"title" : "Whatta proaaa",
"uid" : "4UPmbcaqZKT2NdAAqBahXj4tHYN2"
}
}
},
"users" : {
"4UPmbcaqZKT2NdAAqBahXj4tHYN2" : {
"email" : "rishichandrawawhal@gmail.com",
"profile_picture" : "https://lh4.googleusercontent.com/-vPOIBOxCUpo/AAAAAAAAAAI/AAAAAAAAAFo/SKk9hpOB7v4/photo.jpg",
"username" : "Eena"
}
}
}
```
In case of this JSON, the CLI will generate the following tables:
```sql
users (
_id text not null primary key,
email text,
profile_picture text,
username text
)
posts (
_id text not null primary key,
title text,
body text,
starCount int,
author text,
authorPic text,
uid text
)
user_posts (
_id text not null,
_id_2 text not null,
title text,
body text,
starCount int,
author text,
authorPic text,
uid text
)
```
As we can see, this is not the most efficient of schemas.
- `posts(uid)` can be a foreign key referencing `users(_id)`. `posts(author)` and `posts(authorPic)` can be deleted if `users` and `posts` are related via a foreign key.
- `user_posts` table is obsolete if `posts` and `users` tables are deleted.
To normalize it, here are the steps you must follow:
1. Create the following foreign key constraints:
```sql
ALTER TABLE "posts" ADD CONSTRAINT "posts_users__uid" FOREIGN KEY ("uid") REFERENCES "users"("_id");
ALTER TABLE "posts" DROP COLUMN "author", DROP COLUMN "authorPic";
DROP TABLE "user_posts";
```
To create them, go to the Hasura Console and run the SQL in the `Data > SQL` section.
2. Create the relationships. In the console, go the desired table and add the relationship suggested based on the Foreign key. For example for the `posts` table in the above schema, the suggested relationship would be suggested like:
![suggested](assets/suggested-rel.png)
Click on `Add`. You can name it whatever you like. Lets call it `author` in this case.
![added](assets/added-rel.png)
Similarly, add the suggested relationships for other tables as well.
3. Once you have created relationships, you can start making fancy GraphQL queries like:
```graphql
query {
users (order_by: username_asc){
username
profile_picture
email
posts (where: { starCount: { _gte: 100}}){
title
body
starCount
}
}
}
```
### Duplicates ### Duplicates
@ -304,12 +464,12 @@ In such cases, you have three choices:
1. Use the API as such if you prefer the exact API. 1. Use the API as such if you prefer the exact API.
2. Go to the UI Console and delete the duplicates and normalize the database as you feel fit. 2. Go to the UI Console and delete the duplicates and normalize the database as you feel fit.
3. Use the `--normalize` flag and rerun the migration. In this case, the CLI will detect duplicates and make appropriate relationships between root nodes. (This feature is experimental and needs more test cases to attain stability. Contributions are welcome) 3. Use the `--normalize` flag and rerun the migration. In this case, the CLI will detect duplicates and make appropriate relationships between root nodes. (This feature is experimental and needs more test cases to attain stability. Contributions are welcome)
### Overwrite ### Overwrite
If your database already contains tables with the same name as the root fields of your JSON database, the command will fail. If you want to overwrite the database anyway, you should provide an additional flag "--overwrite". If your database already contains tables with the same name as the root fields of your JSON database, the command will fail. If you want to overwrite the database anyway, you should provide an additional flag `--overwrite`.
## Feedback ## Feedback

Binary file not shown.

After

Width:  |  Height:  |  Size: 42 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 39 KiB

View File

@ -1,10 +1,10 @@
module.exports = { module.exports = {
scores: { f2g_test_scores: {
Rishi: 24, Rishi: 24,
Rikin: 26, Rikin: 26,
Tanmai: 27, Tanmai: 27,
}, },
author: { f2g_test_author: {
someone: { someone: {
one: { one: {
name: 'Rishi', name: 'Rishi',
@ -83,7 +83,7 @@ module.exports = {
}, },
}, },
}, },
articles: { f2g_test_articles: {
first: { first: {
title: 'Rishis article', title: 'Rishis article',
body: "Rishi's article's body", body: "Rishi's article's body",

View File

@ -1,5 +1,5 @@
{ {
"Articles": { "f2g_Articles": {
"A1": { "A1": {
"Title": "Title1", "Title": "Title1",
"Body": "Body1", "Body": "Body1",
@ -55,7 +55,7 @@
} }
} }
}, },
"Authors": { "f2g_test_Authors": {
"AT1": { "AT1": {
"Name": "AName1", "Name": "AName1",
"Age": 11, "Age": 11,
@ -77,7 +77,7 @@
} }
} }
}, },
"Comments": { "f2g_test_Comments": {
"C1": { "C1": {
"Body": "Comment1", "Body": "Comment1",
"Author": { "Author": {

View File

@ -5,4 +5,5 @@ else
F2G_LOG=0 ../bin/run $TEST_HGE_URL --access-key=$TEST_X_HASURA_ACCESS_KEY --db=./data-sets/chinook.json --overwrite --normalize && node verifyChinook.js F2G_LOG=0 ../bin/run $TEST_HGE_URL --access-key=$TEST_X_HASURA_ACCESS_KEY --db=./data-sets/chinook.json --overwrite --normalize && node verifyChinook.js
F2G_LOG=0 ../bin/run $TEST_HGE_URL --access-key=$TEST_X_HASURA_ACCESS_KEY --db=./data-sets/blog.json --overwrite --normalize && node verifyBlog.js F2G_LOG=0 ../bin/run $TEST_HGE_URL --access-key=$TEST_X_HASURA_ACCESS_KEY --db=./data-sets/blog.json --overwrite --normalize && node verifyBlog.js
F2G_LOG=0 ../bin/run $TEST_HGE_URL --access-key=$TEST_X_HASURA_ACCESS_KEY --db=./data-sets/chinook_nested.json --overwrite --normalize && node verifyChinookNested.js F2G_LOG=0 ../bin/run $TEST_HGE_URL --access-key=$TEST_X_HASURA_ACCESS_KEY --db=./data-sets/chinook_nested.json --overwrite --normalize && node verifyChinookNested.js
F2G_LOG=0 ../bin/run $TEST_HGE_URL --access-key=$TEST_X_HASURA_ACCESS_KEY --db=./data-sets/readme-example-1.json --overwrite --normalize && node verifyRE1.js
fi fi

View File

@ -56,17 +56,17 @@ const verifyDataImport = () => {
}), }),
} }
).then(() => { ).then(() => {
console.log(colors.green('✔︎ data-sets/chinook.json: Test passed')); console.log(colors.green('✔︎ data-sets/chinook_nested.json: Test passed'));
process.exit(); process.exit();
}).catch(() => { }).catch(() => {
process.exit(); process.exit();
}); });
} else { } else {
console.log(colors.red('✖ data-sets/chinook.json: Test failed. Unexpected response.')); console.log(colors.red('✖ data-sets/chinook_nested.json: Test failed. Unexpected response.'));
process.exit(); process.exit();
} }
}).catch(e => { }).catch(e => {
console.log(colors.red('✖ data-sets/chinook.json: Test failed. Unexpected response.')); console.log(colors.red('✖ data-sets/chinook_nested.json: Test failed. Unexpected response.'));
console.log(JSON.stringify(e, null, 2)); console.log(JSON.stringify(e, null, 2));
process.exit(); process.exit();

View File

@ -0,0 +1,67 @@
const {query} = require('graphqurl');
const fetch = require('node-fetch');
const colors = require('colors/safe');
const complexQuery = `
query {
f2g_test_Authors (order_by: Name_asc) {
_id
Name
f2g_Articles (order_by: Title_asc, where: { IsUnpublished: { _eq: true}}) {
Title
f2g_test_Comments (order_by: Date_asc) {
Body
Date
}
}
}
}
`;
const verifyDataImport = () => {
query({
query: complexQuery,
endpoint: `${process.env.TEST_HGE_URL}/v1alpha1/graphql`,
headers: {'x-hasura-access-key': process.env.TEST_X_HASURA_ACCESS_KEY},
}).then(response => {
if (
response.data &&
response.data.f2g_test_Authors[0].f2g_Articles.length === 0 &&
response.data.f2g_test_Authors[1].f2g_Articles[0].f2g_test_Comments[0].Body === 'Comment1'
) {
let sqlString = '';
['Articles', 'Authors', 'Comments'].forEach(t => {
sqlString += `drop table public."f2g_test_${t}" cascade;`;
});
fetch(
`${process.env.TEST_HGE_URL}/v1/query`,
{
method: 'POST',
headers: {'x-hasura-access-key': process.env.TEST_X_HASURA_ACCESS_KEY},
body: JSON.stringify({
type: 'run_sql',
args: {
sql: sqlString,
cascade: true,
},
}),
}
).then(() => {
console.log(colors.green('✔︎ data-sets/readme-example-1.json: Test passed'));
process.exit();
}).catch(() => {
process.exit();
});
} else {
console.log(colors.red('✖ data-sets/readme-example-1.json: Test failed. Unexpected response.'));
process.exit();
}
}).catch(e => {
console.log(colors.red('✖ data-sets/readme-example-1.json: Test failed. Unexpected response.'));
console.log(JSON.stringify(e, null, 2));
process.exit();
});
};
verifyDataImport();