REST API by Vad1mo · Pull Request #1299 · SUSE/Portus

Vad1mo · 2017-06-09T23:00:45Z

REST API for Portus to manage Users and Tokens. Furthermore based on this effort here it is possible to add more APIs to Portus.

PR Content and Features:

CRUD Users and their application tokens via a REST PAI
Swagger Spec/OpenAPI
Documentation

mssola · 2017-06-10T07:48:22Z

First of all, thanks a lot for this 👍

Now, you may want to take a look at what we are doing here: #1289. There's a lot of stuff on this PR, but take into consideration:

The usage of PORTUS-AUTH in app/controllers/application_controller.rb. This is inspired by what Gitlab is doing, and it allows remote clients to make API calls.
The app/controllers/tags_controller.rb file now implements the show method, which is the first one providing API support.

Now, this is far from perfect, so I encourage you to innovate upon these ideas. I also thought of using something like grape, but so far I've used good ol' Rails as I'm already used to it (also notice all the ugly as_json methods, that I'd like to get rid off with a proper serializer or other solution...).

Vad1mo · 2017-06-10T08:06:25Z

I saw some of the work in the branch. We will take PORTUS-AUTH into consideration. @mssola when do you plan to merge #1289 into master?
I like the grape approach. We also see this as an viable option and thinking about using it.

Vad1mo · 2017-06-10T12:43:32Z

It would be nice if someone could review the API Docs f1999b7 It would be nice if @mssola or someone else would take some time to review the API concept.

mssola · 2017-06-12T06:52:28Z

I saw some of the work in the branch. We will take PORTUS-AUTH into consideration. @mssola when do you plan to merge #1289 into master?

Both me and @vitoravelino have been busy with other stuff, and it will probably take a while, since we have to review the code, and apply more testing... I'd say I can break away the code for managing PORTUS-AUTH in another PR, so you can use it.

I like the grape approach. We also see this as an viable option and thinking about using it.

I have to admit that I've never used it, so this is something I'm not really sure. It needs some exploration from my part 😄

It would be nice if someone could review the API Docs f1999b7 It would be nice if @mssola or someone else would take some time to review the API concept.

Left some comments. In principle it looks good to me 👍

mssola · 2017-06-21T13:53:36Z

@Vad1mo ping me whenever this is ready to be reviewed.

Vad1mo · 2017-06-21T14:04:46Z

@mssola we are fixing the travis build and adding some documentation next to the swagger spec. The code is basically ready for an review.

fyi: you can get the swagger spec localhost:3000/api/swagger_doc

mssola · 2017-06-21T14:25:28Z

@Vad1mo ok, I'll wait also for the swagger documentation if you don't mind 😉

andrew2net · 2017-06-22T08:36:56Z

@mssola Hello,
I'm trying to pass travis. Running command, which got an error on travis:

bundle exec rspec spec packaging/suse/portusctl/spec

got

...
rpc error: code = 2 desc = oci runtime error: exec failed: container_linux.go:247: starting container process caused "exec: \"mysqladmin\": executable file not found in $PATH"

Command 'mysqladmin --count=1 --connect-timeout=1 --host=integration_db -u root --password=portus ping' failed! Waiting 5 seconds...
...

can you help me find cause?

mssola · 2017-06-22T08:57:21Z

@andrew2net have you tried without running the integration tests ? (they might be broken...):

$ bundle exec rspec spec packaging/suse/portusctl/spec --tag ~integration

andrew2net · 2017-06-22T09:11:23Z

@mssola I haven't. traied now and all tests passed. Does it need to pass integration tests for successfull travis checkung?

mssola · 2017-06-22T09:35:56Z

@andrew2net no. Travis does not run integration tests for Portus.

Vad1mo · 2017-06-22T23:03:57Z

@mssola you can take a look now at the PR

mssola

Left some comments. Thanks a lot for all your hard work 👏

mssola · 2017-06-23T11:15:25Z

 /spec/integration/fixtures/data
-/node_modules
+/node_modules
+.idea


Remove this line

mssola · 2017-06-23T11:16:39Z

  Enabled: false

+Style/FrozenStringLiteralComment:
+  Enabled: false


This wasn't an issue before 😕 If you call it with bundle exec there shouldn't be any problem no ?

@mssola with ruby 2.3 rubocop reports offense Missing frozen string literal comment. on each rb file

mssola · 2017-06-23T11:19:07Z

@@ -1 +1 @@
-2.1.2
+2.3.3


Unrelated to this PR.

well, grape requires to go to minimum 2.3.

I'll have to check about this requirement...

mssola · 2017-06-23T11:20:19Z


  # Stable versions
-  - 2.1.10
-  - 2.2.6


The changes on this file are unrelated to this PR. Moreover, why should we stop testing these versions ? (maybe we could upgrade the 2.2.6 to 2.2.7)

ruby 2.3 is required

mssola · 2017-06-23T11:21:12Z

+gem "grape-swagger"
+gem "grape-swagger-entity"
+gem "grape-swagger-representable"
+gem "hashie-forbidden_attributes"


I'm not sure about this gem... Do you really need mass assignment ?

@mssola ruby-grape uses it https://github.com/ruby-grape/grape#rails

mssola · 2017-06-23T11:34:29Z

@@ -0,0 +1,20 @@
+class VulnerabilitiesController < ApplicationController


This is from another PR. Please, don't include it...

mssola · 2017-06-23T11:39:18Z

@@ -0,0 +1,191 @@
+module GrapeAPI


Two things:

There's functionality already existing in app/controllers/application_tokens_controller.rb for this. Could we merge the code somehow ? (either changes the frontend to use this API instead of the controller, thus removing the controller, or creating methods that both the controller and the API use)

Could you add more documentation ?

@mssola the functionality slightly different. API can create tokens for any user, not only for current, and returns plain_token. I think changing the frontend to use the API is preferred.
Will add more doc.

The functionality is moved to model.

mssola · 2017-06-23T11:43:06Z

+  end
+
+  class RootAPI < Grape::API
+    # version 'v1', using: :path


why not? 😀

indeed 😄

mssola · 2017-06-23T11:43:39Z

+        error!("Authorization fails.", 403) unless user.admin
+      end
+
+      def current_user


This already exists in the application_controller.rb. We should be using the same code and share it somehow.

Moved it to concern auth_from_token.

mssola · 2017-06-23T11:45:15Z

+        title:         "Portus user API",
+        description:   "CRUD users and user's application tokens API",
+        contact_name:  "Anldrei Kislichenko",
+        contact_email: "andrei.kislichenko@gmail.com"


I'd prefer:

Contact name: "Portus authors" (which would include you 😉)

Contact email: "portus-dev@googlegroups.com"

@mssola ok, I don't mind 😄

mssola · 2017-07-18T11:27:18Z

Note that we have just merged #1289, and so there are quite some conflicts now. Also, make sure to rebase against master.

Vad1mo · 2017-07-20T06:09:07Z

@mssola IMHO, we are ready now, for the review.

mssola

In general it looks good to me. Just fix my comments 😉 I still have to test it locally more deeply though ...

mssola · 2017-08-04T14:58:20Z

+    username, password = auth.split(":")
+    user = User.find_by(username: username)
+    if user && user.application_token_valid?(password)
+


Remove this empty line. Maybe you could add this empty line before the if statement actually.

mssola · 2017-08-04T14:58:56Z

@@ -0,0 +1,16 @@
+module AuthFromToken


Please, add documentation

mssola · 2017-08-04T14:59:12Z

+module AuthFromToken
+  extend ActiveSupport::Concern
+
+  def authenticate_user_from_authentication_token!


Add documentation please 😉

mssola · 2017-08-04T15:10:16Z

+      expose :id, documentation: { type: Integer, desc: "User id" }
+      expose :username, documentation: { type: String, desc: "User name" }
+      expose :email, documentation: { type: String, desc: "E-mail" }
+      expose :sign_in_count, documentation: { type: Integer }


Is this really needed ?

sign_in_count we can skip. The others we would like to keep. We want to use the API to perform CRUD operations on the on the user, so we can decide what users to create and maybe remove users automatically that don't use their account. However the last use case is no yet relevant for us right now.

mssola · 2017-08-04T15:10:52Z

+      expose :sign_in_count, documentation: { type: Integer }
+      expose :current_sign_in_at, documentation: { type: DateTime }
+      expose :last_sign_in_at, documentation: { type: DateTime }
+      expose :current_sign_in_ip, :last_sign_in_ip


Not sure about these two, since it's not that useful imho

ok, not really relevant.

mssola · 2017-08-04T15:22:12Z

+
+        route_param :id, type: Integer do
+          resource :application_tokens do
+


Remove empty line

mssola · 2017-08-04T15:24:06Z

+
+            params do
+              requires :application, documentation: { desc: "Application name" }
+            end


Add an empty line after the end line.

mssola · 2017-08-04T15:24:17Z

+
+          params do
+            requires :id, documentation: { desc: "Token id" }
+          end


mssola · 2017-08-04T15:25:55Z

+              only:  [:display_name],
+              using: API::Entities::Users.documentation.slice(:display_name)
+          end
+        end


mssola · 2017-08-04T15:26:11Z

+              only:  [:display_name],
+              using: API::Entities::Users.documentation.slice(:display_name)
+          end
+        end


mssola

Some comments after using it:

There should be a way to return a 404 when the path does not exist. For example I typed /api/users, and I got an HTML page, instead of simply a not found error. Maybe in production this is already handled?

Besides this, when I performed:

curl -X DELETE \
     -H "PORTUS-AUTH: mssola:KXsgWZQyDq8rMoy-hHaD" \
    https://registry.mssola.cat/api/v1/users/2/application_tokens/2

I got:

{
  "error": "Internal server error: Grape::Exceptions::MethodNotAllowed"
}

Overall really nice work. Thanks a lot for your patience 👏

mssola · 2017-08-08T10:29:00Z

+      },
+      security:             [api_key: []],
+      info:                 {
+        title:         "Portus user API",


Argh, overlooked this, sorry ... It should be "Portus API" instead since we will extend it to other endpoints as well...

mssola · 2017-08-08T10:33:49Z

+      security:             [api_key: []],
+      info:                 {
+        title:         "Portus user API",
+        description:   "CRUD users and user's application tokens API",


Same here. It should be something like this: "Portus CRUD API" (or better 😅 )

mssola · 2017-08-08T13:14:28Z

+            [401, "Authentication fails."],
+            [403, "Authorization fails."]
+          ],
+          consumes: ["application/x-www-form-urlencoded"]


Maybe this can be done in another PR, but would it be too difficult to also support JSON ? It would be simpler than with an encoded form. See the curl command I had to use:

curl -X POST \ -H "PORTUS-AUTH: mssola:KXsgWZQyDq8rMoy-hHaD" \ -F "user[username]=another" -F "user[email]=asd@example.org" -F "user[password]=12341234" \ https://registry.mssola.cat/api/v1/users

Maybe this could work too (so without the user[...] part):

curl -X POST \ -H "PORTUS-AUTH: mssola:KXsgWZQyDq8rMoy-hHaD" \ https://registry.mssola.cat/api/v1/users?username=another&email=asd@example.org&password=12341234

Will add JSON

curl -X POST "http://localhost:3005/api/v1/users" \ -H "accept: application/json" \ -H "Content-Type: application/json" \ -H "Portus-Auth: andrei:xzgZj6toUiuM2h49zA7z" \ -d '{"user":{"username":"another","email":"asd@example.org","password":"12341234"}}'

mssola · 2017-08-08T13:17:51Z

+            [403, "Authorization fails."],
+            [404, "Not found."]
+          ],
+          consumes: ["application/x-www-form-urlencoded"]


andrew2net · 2017-08-08T18:57:01Z

@mssola API with GRAPE processes requests if the path matches any of endpoints. If not then the request is ignored by GRAPE and the app processes it and returns No route matches error in development as HTML if route not exists.
Will add interception of Grape::Exceptions::MethodNotAllowed error.

mssola

If I submit a malformed JSON (create user), I receive and HTML. It should instead return a proper error response (in this case this is a 400 Bad Request).

We are getting really close, nice work!

mssola · 2017-08-10T14:58:21Z

+        expect(JSON.parse(response.body)["errors"]).not_to be_nil
+      end
+
+      it "returs user error" do


Typo: "returs" -> "returns"

mssola

LGTM

Rebase all this to the master branch, and squash commits into a set of commits (or a single commit even)

Again, thanks a lot for all this. You rock 👏

Signed-off-by: Vadim Bauer <bauer.vadim@gmail.com>

mssola

LGTM 👏

mssola suggested changes Jun 23, 2017

View reviewed changes

Vad1mo changed the title ~~WIP: REST API~~ REST API Jul 19, 2017

mssola added this to the Release 2.3 milestone Aug 2, 2017

mssola suggested changes Aug 4, 2017

View reviewed changes

mssola reviewed Aug 8, 2017

View reviewed changes

mssola mentioned this pull request Aug 8, 2017

Add a Help page with more resources #1372

Open

mssola reviewed Aug 10, 2017

View reviewed changes

mssola previously approved these changes Aug 11, 2017

View reviewed changes

Vad1mo dismissed mssola’s stale review via 70e9a4d August 11, 2017 22:52

Vad1mo force-pushed the api branch from 96cc77d to 70e9a4d Compare August 11, 2017 22:52

Portus REST API for creating and managing users and tokens.

28f77d3

Signed-off-by: Vadim Bauer <bauer.vadim@gmail.com>

Vad1mo force-pushed the api branch from 70e9a4d to 28f77d3 Compare August 11, 2017 22:56

mssola approved these changes Aug 14, 2017

View reviewed changes

mssola merged commit 586636b into SUSE:master Aug 14, 2017

mssola mentioned this pull request Aug 14, 2017

js: Namespaces#index refactoring w/ Vue #1371

Merged

8 tasks

mssola mentioned this pull request Nov 2, 2017

Consolidate the REST API #1500

Open

Vad1mo deleted the api branch February 6, 2018 21:53

		@@ -0,0 +1,20 @@
		class VulnerabilitiesController < ApplicationController


		route_param :id, type: Integer do
		resource :application_tokens do

		@@ -1 +1 @@
		2.1.2
		2.3.3

Conversation

Vad1mo commented Jun 9, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mssola commented Jun 10, 2017

Uh oh!

Vad1mo commented Jun 10, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Vad1mo commented Jun 10, 2017

Uh oh!

mssola commented Jun 12, 2017

Uh oh!

mssola commented Jun 21, 2017

Uh oh!

Vad1mo commented Jun 21, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mssola commented Jun 21, 2017

Uh oh!

andrew2net commented Jun 22, 2017

Uh oh!

mssola commented Jun 22, 2017

Uh oh!

andrew2net commented Jun 22, 2017

Uh oh!

mssola commented Jun 22, 2017

Uh oh!

Vad1mo commented Jun 22, 2017

Uh oh!

mssola left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

andrew2net Jun 23, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

andrew2net Jun 23, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mssola commented Jul 18, 2017

Vad1mo commented Jun 9, 2017 •

edited

Loading

Vad1mo commented Jun 10, 2017 •

edited

Loading

Vad1mo commented Jun 21, 2017 •

edited

Loading

andrew2net Jun 23, 2017 •

edited

Loading

andrew2net Jun 23, 2017 •

edited

Loading

Vad1mo commented Jul 20, 2017 •

edited

Loading

Vad1mo Aug 4, 2017 •

edited

Loading