That can be the case if the router is set to automatically assign to any service type, because then it can connect to 2G/3G instead of LTE.

Solution1 :  Change the Connection Type to 4G Only.

To configure  Mobile Connection Type  for 4G Router:

1. Launch a web browser and open login page of your 4G Router.

2. After login, Select  Setup >Internet Settings.

3. Find and change the Mobile Connection Type to "4G ONLY"

4. Apply the settings
You have successfully configured  Mobile Connection Type  for 4G Router.

Solution2 :   Please check the APN and Band that are currently in use and set it to the best value.

APN or Band automatically detected by router may not be suitable for your environment   
Check the settings from your service provider or on your phone and then set  the router with the values.

To configure  4G BAND  for 4G Router:

1. Launch a web browser and open login page of your 4G Router.

2. After login, Select  Setup >Internet Settings.

3. Find and change the 4G BAND to the value you want

4. Apply the settings and you have successfully configured   4G BAND  for 4G Router.