Look at the current sense signal and MOSFET voltage and observe the wave shapes. If the converter is near light load, the transformer will be in discontinuous conduction mode. That means the transformer current starts at zero and LINEARLY ramps up to a peak value before shutting off. If you see a large initial current spike at turn on, more than can be filtered with 1/2 use of RC time constant filtering, either there is a transformer phasing problem or there is some other winding fault / loading condition.
Above some minimum load, the transformer may enter into continuous conduction mode, with an initial step in the current waveform. If at any time you see the linear ramp portion begin to curl upward exponentially, as you slowly increase loading, your transformer is beginning to saturate. It could be due to transformer reset problems, or you may simply be trying to get more power out than the core will support with a given gap / permeability.
This IC is somewhat new and unique and it can also create issues. Because it is newer generation device, requiring minimum external parts, it also has internal automatic features, not found in earlier generation IC's. These features can complicate debugging and initial bring-up
If the current sense resistor voltage at pin 4 exceeds 0.8V, the IC will automatically go into current limiting. If 4 consecutive over-current faults are detected, the IC will shut down for 130 msec before restarting, creating a current limit burst mode condition.
If pin 3 is brought above 3V, the over-voltage / overpower protection feature will be activated. It is probably best to ground pin 3 while debugging.
However, do not ground or disable pin 4, because the current sense signal at pin 4 provides both current limit protection and current feedback for current mode control.
A convenient strategy for debugging is to run the IC housekeeping pin 5 input from a separate lab supply. Usually, you will have to bring it up to the starting threshold, between 16 and 20 volts to get it to turn on and see a gate drive waveform on your MOSFET. Once it turns on, you can reduce the lab housekeeping supply down to 12 to 15 volts. The IC will not shut-off until this HK voltage drops below the 8.2 to 9.4 volt shut-off threshold.
However, the chip also has a frequency flyback feature not encountered with older generation designs. This can complicate the debugging, bring-up procedure. You may need to ground the feedback signal temporarily to disable this, so that the IC will run at full frequency when you first apply lab housekeeping power.
You can then apply main lab power to the converter input separately, since your control chip is already running and providing gate drive, independently of the power stage and any problems it might have. Bring the main input voltage up slowly while observing the voltage and current waveforms for signs of distress. You can vary the input voltage and keep load constant, or vary the load with whatever input voltage you chose. This is pretty much standard procedure for debugging any power converter, whether it is 5 watts for 5 kW.
If your feedback signal is disabled, you can initially debug the flyback by regulating the input voltage and output load to control the output voltage. You have to be careful to monitor the output voltage under these conditions so that it does not "run-away" with either two much input voltage or not enough output loading.
You need to know in advance however, what your primary voltage and current waveforms are supposed to look like. Abraham Pressman's classic 2nd edition of his book on Power Supply Design is a good start. An accurate SPICE simulation will also give you some clues if you know how to model the design.