...

The SAML providers will have to have specific users with specific fields to be set appropriately for the accompanying test. Rather than creating that data upfront, it is each test suites responsibility to create the provider This can be accomplished a couple different ways. That data can be created upfront in the SAML provider or each test suite can be responsible to create its own test data prior to each test. Since If the data will need to be verified and possibly reset before each test run, there doesn’t not seem to be a reason in the SAML provider is static, that is it won’t be updated during a test run and be the same from test run to test run. It is suggested to create that data up frontupfront. However, this could differ from provider to provider, if the test requires the update of the SAML data, for instance to test if SAML fields update properly. Then the data should be either be verified or deleted and recreated before the test run.

If test data is required to be added to the Mattermost database. That data can be added in the following file - ../mattermost-server/cmd/mattermost/commands/sampledata.go.

...

• SAML User Login - New user

• SAML User Login - Existing user

• SAML Guest Login - New user

• SAML Guest Login - Existing user

• SAML Admin Login - New user

• SAML Admin Login - Existing user

Once the base tests have been completed, the manual release tests will be next.
https://docs.google.com/spreadsheets/d/18nQEYeHRQCkHPRmiDZLd-z8DyMg1TlDfDu-c0qlK8Vs/edit#gid=932310946

Required SAML APIs

Create User - create a new user in SAML Server

...

• Config Settings

  • SAMLSettings.Enable = true

• API Functions (separate file)

  • CreateUser

  • UpdateUser

  • DeactivateUser

  • RetrieveUser

  • ResetUser

• Properties

  • signin-submit textbox name

  • SAMLProviderLogin()

Known Issues
These tests may have to run as separate suites from the current tests. The reason is that these tests will require the cypress.json file to have the following setting. "chromeWebSecurity": false

1. "chromeWebSecurity": false
   
   For SAML we have to turn off chromeWebSecurity to allow us to go to another website, otherwise Cypress will not allow this to protect against cross-site scripting. This config setting will need to be reset before/after our tests. From the POC testing, it appears this setting cannot be set at runtime and we don’t want this setting = false for most testing

...

1. . These tests may therefore need to be a separate test suite.
   

2. ExperimentalSettings.UseNewSAMLLibrary
   

...

1. 
   For completeness, all of the tests should be run twice, once using the existing SAML Library and one using the New SAML Library. However, that configuration setting is only loaded on server start, it does not change on the fly. Therefore, the base configuration will need to be updated and the server restarted. Running against a “true” setting may be all that is required for automated testing. As long as we can reset the setting and run the tests manually.
   

2. Certificates
   
   Certificates will need to be created, stored and installed as part of configuration setup. We don’t want new certificates for each run as that would require setting up providers prior to each test run.
   

Ad-hoc servers - Known Issues
Supporting ad-hoc servers brings its own set of tasks.

• Certificates - will need loaded

• SAML Provider settings may need updated, for server name to callback.
  

Proof of Concept

In order to ensure that the above specification is at least feasible, a proof of concept was developed.

...