Hello Dison,
Using a breadboard is not very suitable when it comes to power circuits therefore i really suggest you to make a PCB layout.
Also let me give you some advices when you create the layout.
1. The most important layout rule is to keep the AC current loops as small as possible. In a BOOST regulator the primary switching loop consists of the output capacitor, diode, MOSFET and sensing resistor. Minimizing the area of this loop reduces the stray inductances and minimizes noise and possible erratic operation.
2. The current sensing circuit in current mode devices can be easily effected by switching noise. This noise can cause duty cycle jitter which leads to increased spectral noise. The sensing resistor should be connected to the ISEN pin with a separate trace made as short as possible. Route this trace away from the inductor and the switch node. For the voltage loop, keep the feedback resistors close to the IC and run a trace as close as possible to the positive side of the output capacitor. As with the ISEN line, the FB line should be routed away from the inductor and the switch node. These measures minimize the length of high impedance lines and reduce noise pickup.
3. A ground plane in the printed circuit board is recommended as a means to connect the quiet end (input voltage ground side) of the input filter capacitor to the output filter capacitors and the PGND pin of the controller. Connect all the low power ground connections directly to the regulator AGND. Connect the AGND and PGND pins together through a copper area covering the entire underside of the device. Place several vias in this underside copper area to ground plane. If a via is needed to connect the sensing resistor to the ISEN pin, then place that via in the inner side of the sensing resistor such that no current flow occurs. Place several vias from the ground side of the output capacitor(s) to ground place, that will minimize the path for AC current. The PGND and AGND pins have to be connected to the same ground very close to the IC. To avoid ground loop currents attach all the grounds of the system only at one point. A general rule of thumb is to maximize the low impedance area (VIN,VOUT, GND).
4. Use thicker traces for the gate signal from the DR pin to the gate of the MOSFET.
Furthermore, could you please share your final design where VIN = 24V and VOUT=150V@150mA? I would like to review it.
Regards,
Giuseppe