Register cluster handlers immediately after starting the server

Description

`NewServer` starts the cluster. But the handlers aren't really registered until InitServer is called, which doesn't happen until a first request hits in web/handlers.go or in app/websocket_router.go.

We should fix it to register as soon as the cluster is started.

QA Test Steps

An HA environment is needed to test this.

  • Spin up an HA environment

  • Check the server logs via terminal (not from system console w/o actually logging in the servers)

  • Make sure there are no errors regarding to cluster handlers like below:
    `
    {"level":"error","ts":1617019138.2336092,"caller":"cluster/cluster.go:224","msg":"ClusterInterfaceImpl.handleClusterMessage no handler for message","message":"{\"event\":\"update_status\",\"data\":\"{\\\"user_id\\\":\\\"3qdpdtaaipf4xrbrgxur3jntqh\\\",\\\"status\\\":\\\"online\\\",\\\"manual\\\":false,\\\"last_activity_at\\\":1617019138234,\\\"active_channel\\\":\\\"3fw3c4emcbffmgk3whoyozwarh\\\"}\"}"}
    `

  • Login to server as system admin, start some jobs and make sure everything is working as expected.

Activity

Show:
Saturnino Abril
7 days ago

Tested and passed on 5.34.0-rc2

Saturnino Abril
April 7, 2021, 5:52 AM

Tested against 5.31.3-rc1 HA servers and passed.

  • see logs at

Amy Blais
April 6, 2021, 7:11 PM
Neill Collie
March 30, 2021, 3:34 PM

screenshots showing

initial issue, then logging to to second node, then back to initial node to see issue resolved.

Neill Collie
March 30, 2021, 3:33 PM

A customer recently reported this issue on upgrade to 5.31.
I’ve replicated on 5.33.


Until a request to web or websocket request is received, no handler is available for example, when listing plugins:

{"level":"error","ts":1617104966.997595,"caller":"mlog/log.go:232","msg":"Unable to get plugin statuses from the cluster.","path":"/api/v4/plugins/statuses","request_id":"5wazp7ocsjns3jxtpmxt4xzgwe","ip_addr":"192.168.0.29","user_id":"btrpip9t6idemq3mjpeiak9kch","method":"GET","err_where":"GetClusterPluginStatuses","http_code":500,"err_details":": Plugins have been disabled. Please check your logs for details., "}

when you then login to the second node, the handler seems to be fired up and plugins/statuses beings responding as expected.

Issue seems to affect the UI in that installed plugins are not visible, and job scheduling from dedicated jobs servers.

 




Done

Mana

None

Assignee

Ibrahim Acikgoz

QA Assignee

Saturnino Abril

Reporter

Agniva De Sarker

Epic Link

None

Mattermost Team

Sustained Engineering

Sprint

None

QA Testing Areas

None

GitHub Issue

None

Components

None

Severity

None