You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-13 - 20:48

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #8, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack8slot6s.osadl.org (updated Thu Nov 13, 2025 12:43:44)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
9856992121,0cyclictest26475-21fwupd09:06:192
9856991918,1cyclictest66850irq/125-lan11:51:512
9856991918,1cyclictest66850irq/125-lan11:00:142
9856991918,1cyclictest66850irq/125-lan10:30:382
9856991918,1cyclictest66850irq/125-lan08:50:142
9856991818,0cyclictest66850irq/125-lan12:15:162
9856991818,0cyclictest66850irq/125-lan11:45:132
9856991818,0cyclictest66850irq/125-lan11:44:242
9856991818,0cyclictest66850irq/125-lan10:46:492
9856991818,0cyclictest66850irq/125-lan10:15:162
9856991818,0cyclictest66850irq/125-lan09:36:142
9856991817,1cyclictest66850irq/125-lan12:30:432
9856991817,1cyclictest66850irq/125-lan12:28:372
9856991817,1cyclictest66850irq/125-lan11:30:562
9856991817,1cyclictest66850irq/125-lan11:21:512
9856991817,1cyclictest66850irq/125-lan11:17:472
9856991817,1cyclictest66850irq/125-lan11:10:152
9856991817,1cyclictest66850irq/125-lan10:37:192
9856991817,1cyclictest66850irq/125-lan10:21:012
9856991817,1cyclictest66850irq/125-lan10:21:012
9856991817,1cyclictest66850irq/125-lan10:10:102
9856991817,1cyclictest66850irq/125-lan09:25:062
9856991817,1cyclictest66850irq/125-lan09:11:062
9856991817,1cyclictest66850irq/125-lan08:45:202
9856991717,0cyclictest66850irq/125-lan12:07:142
9856991717,0cyclictest66850irq/125-lan12:00:122
9856991717,0cyclictest66850irq/125-lan10:55:482
9856991717,0cyclictest66850irq/125-lan10:08:042
9856991717,0cyclictest66850irq/125-lan10:08:032
9856991717,0cyclictest66850irq/125-lan09:49:422
9856991717,0cyclictest66850irq/125-lan09:42:042
9856991717,0cyclictest66850irq/125-lan09:21:242
9856991717,0cyclictest66850irq/125-lan09:15:432
9856991717,0cyclictest66850irq/125-lan08:30:222
9856991717,0cyclictest66850irq/125-lan08:10:222
9856991717,0cyclictest66850irq/125-lan07:50:162
9856991716,1cyclictest8722-21kworker/2:3+events09:54:042
9856991716,1cyclictest66850irq/125-lan12:35:342
9856991716,1cyclictest66850irq/125-lan12:21:092
9856991716,1cyclictest66850irq/125-lan12:14:532
9856991716,1cyclictest66850irq/125-lan11:25:172
9856991716,1cyclictest66850irq/125-lan11:07:182
9856991716,1cyclictest66850irq/125-lan10:00:132
9856991716,1cyclictest66850irq/125-lan09:55:142
9856991716,1cyclictest66850irq/125-lan09:00:132
9856991616,0cyclictest66850irq/125-lan10:40:242
9856991616,0cyclictest66850irq/125-lan08:00:162
9856991616,0cyclictest66850irq/125-lan07:55:132
9856991616,0cyclictest66850irq/125-lan07:45:212
9856991616,0cyclictest66850irq/125-lan07:20:212
9856991615,1cyclictest66850irq/125-lan11:56:242
9856991615,1cyclictest66850irq/125-lan11:35:052
9856991615,1cyclictest66850irq/125-lan10:25:182
9856991615,1cyclictest66850irq/125-lan08:25:162
9856991615,1cyclictest66850irq/125-lan08:05:212
9856991615,1cyclictest66850irq/125-lan07:35:162
9856991615,1cyclictest66850irq/125-lan07:30:112
9856991515,0cyclictest66850irq/125-lan09:33:252
9856991514,1cyclictest66850irq/125-lan10:52:202
9856991514,1cyclictest66850irq/125-lan07:25:122
9856991414,0cyclictest66850irq/125-lan07:40:172
9856991413,1cyclictest66850irq/125-lan08:15:212
9856991413,1cyclictest66850irq/125-lan07:10:192
9856991313,0cyclictest66850irq/125-lan08:40:142
9856991313,0cyclictest66850irq/125-lan08:20:122
9856991212,0cyclictest66850irq/125-lan08:55:122
9856991110,1cyclictest66850irq/125-lan08:35:192
98619998,1cyclictest771-21snmpd09:49:573
98619998,1cyclictest771-21snmpd09:14:543
98619998,1cyclictest771-21snmpd08:40:463
98619998,1cyclictest524-21avahi-daemon10:23:093
98619998,1cyclictest524-21avahi-daemon10:23:083
98619998,1cyclictest31280-21fschecks_time09:50:123
98569999,0cyclictest66850irq/125-lan07:15:182
98499998,1cyclictest771-21snmpd07:49:261
98499998,1cyclictest769-21lldpd09:22:031
98499998,1cyclictest524-21avahi-daemon11:20:481
98439998,1cyclictest771-21snmpd12:24:300
98619988,0cyclictest771-21snmpd12:26:383
98619988,0cyclictest771-21snmpd11:56:193
98619988,0cyclictest771-21snmpd10:14:103
98619988,0cyclictest771-21snmpd07:40:383
98619988,0cyclictest769-21lldpd12:32:423
98619988,0cyclictest769-21lldpd12:20:423
98619988,0cyclictest769-21lldpd12:09:403
98619988,0cyclictest769-21lldpd11:44:213
98619988,0cyclictest769-21lldpd09:31:533
98619987,1cyclictest517-21systemd-resolve10:35:033
98619986,2cyclictest771-21snmpd12:17:303
98619986,2cyclictest771-21snmpd11:37:323
98619986,2cyclictest771-21snmpd11:22:283
98619986,2cyclictest771-21snmpd10:44:593
98619986,2cyclictest771-21snmpd08:51:493
98619984,3cyclictest4582-21ntp_states09:20:183
98499988,0cyclictest771-21snmpd11:08:341
98499988,0cyclictest771-21snmpd10:21:591
98499988,0cyclictest771-21snmpd10:21:581
98499988,0cyclictest771-21snmpd09:50:331
98499988,0cyclictest769-21lldpd09:44:051
98499988,0cyclictest769-21lldpd07:10:241
98499987,1cyclictest771-21snmpd08:42:121
98499987,1cyclictest769-21lldpd07:54:481
98499987,1cyclictest524-21avahi-daemon10:07:441
98499987,1cyclictest524-21avahi-daemon10:07:441
98499987,1cyclictest524-21avahi-daemon09:15:401
98499987,1cyclictest524-21avahi-daemon09:11:451
98499987,1cyclictest255-20systemd-journal10:19:561
98499986,2cyclictest771-21snmpd12:11:201
98499986,2cyclictest771-21snmpd12:08:451
98499986,2cyclictest771-21snmpd07:59:291
98499984,4cyclictest255-20systemd-journal12:31:401
98439988,0cyclictest771-21snmpd12:36:280
98439988,0cyclictest771-21snmpd08:36:240
98439988,0cyclictest771-21snmpd08:22:330
98439988,0cyclictest771-21snmpd08:11:560
98439988,0cyclictest769-21lldpd11:38:000
98439988,0cyclictest769-21lldpd11:11:160
98439988,0cyclictest769-21lldpd11:00:400
98439988,0cyclictest769-21lldpd09:35:270
98439988,0cyclictest769-21lldpd07:50:590
98439987,1cyclictest771-21snmpd10:55:530
98439987,1cyclictest771-21snmpd10:12:500
98439987,1cyclictest771-21snmpd07:59:560
98439987,1cyclictest769-21lldpd11:26:560
98439987,1cyclictest769-21lldpd09:40:170
98439987,1cyclictest769-21lldpd08:45:430
98439987,1cyclictest524-21avahi-daemon12:15:420
98439987,1cyclictest25961-21tail11:30:200
98439986,2cyclictest771-21snmpd10:16:280
98439986,2cyclictest771-21snmpd09:09:250
98439986,2cyclictest771-21snmpd08:02:090
98439985,2cyclictest24572-21grep07:45:150
98619977,0cyclictest771-21snmpd11:02:243
98619977,0cyclictest771-21snmpd10:25:133
98619977,0cyclictest771-21snmpd07:55:273
98619977,0cyclictest769-21lldpd11:12:003
98619977,0cyclictest769-21lldpd10:32:313
98619977,0cyclictest769-21lldpd08:59:133
98619977,0cyclictest769-21lldpd08:01:203
98619976,1cyclictest771-21snmpd10:51:053
98619976,1cyclictest771-21snmpd09:01:043
98619976,1cyclictest10535-21cpuspeed_turbos08:30:143
98619975,2cyclictest771-21snmpd12:37:583
98619975,2cyclictest771-21snmpd12:02:533
98619975,2cyclictest771-21snmpd10:00:303
98619975,2cyclictest25416-21tail10:55:153
98619973,3cyclictest255-20systemd-journal11:16:023
98619973,3cyclictest255-20systemd-journal11:05:223
98499977,0cyclictest771-21snmpd11:40:071
98499977,0cyclictest771-21snmpd10:37:001
98499977,0cyclictest771-21snmpd08:24:391
98499977,0cyclictest769-21lldpd11:45:491
98499977,0cyclictest769-21lldpd11:01:201
98499976,1cyclictest771-21snmpd11:26:151
98499976,1cyclictest771-21snmpd09:58:271
98499976,1cyclictest771-21snmpd09:36:321
98499976,1cyclictest771-21snmpd07:40:341
98499976,1cyclictest524-21avahi-daemon11:32:131
98499976,1cyclictest255-20systemd-journal10:41:331
98499976,1cyclictest2435-21cpuspeed_turbos08:10:121
98499976,1cyclictest16004-21cpuspeed_turbos11:55:131
98499975,2cyclictest771-21snmpd12:26:111
98499975,2cyclictest771-21snmpd11:51:231
98499975,2cyclictest771-21snmpd08:27:511
98499973,3cyclictest2710-21fw_forwarded_lo10:30:151
98499973,3cyclictest23574-21ssh12:37:111
98439977,0cyclictest771-21snmpd09:10:070
98439977,0cyclictest771-21snmpd08:56:040
98439977,0cyclictest771-21snmpd08:16:190
98439977,0cyclictest769-21lldpd10:25:210
98439977,0cyclictest769-21lldpd09:51:410
98439977,0cyclictest769-21lldpd07:38:110
98439976,1cyclictest771-21snmpd10:07:520
98439976,1cyclictest771-21snmpd10:07:520
98439976,1cyclictest771-21snmpd09:32:570
98439976,1cyclictest771-21snmpd08:31:010
98439976,1cyclictest771-21snmpd07:32:160
98439976,1cyclictest517-21systemd-resolve10:21:070
98439976,1cyclictest517-21systemd-resolve10:21:060
98439976,1cyclictest517-21systemd-resolve10:01:410
98439975,2cyclictest771-21snmpd11:50:140
98439975,2cyclictest771-21snmpd10:52:240
98439975,2cyclictest771-21snmpd09:04:530
98439975,2cyclictest771-21snmpd07:44:010
98439974,0cyclictest0-21swapper/007:20:120
98619966,0cyclictest771-21snmpd09:36:593
98619966,0cyclictest769-21lldpd07:23:093
98619966,0cyclictest524-21avahi-daemon07:10:583
98619965,1cyclictest771-21snmpd09:55:313
98619964,2cyclictest771-21snmpd10:45:273
98619964,2cyclictest771-21snmpd10:18:373
98619963,3cyclictest550-21NetworkManager09:16:443
98499966,0cyclictest771-21snmpd08:03:111
98499966,0cyclictest771-21snmpd07:27:021
98499966,0cyclictest769-21lldpd08:16:021
98499966,0cyclictest769-21lldpd07:32:131
98499966,0cyclictest524-21avahi-daemon10:27:201
98499965,1cyclictest771-21snmpd07:18:341
98499965,1cyclictest6743-21cat12:20:121
98499965,1cyclictest524-21avahi-daemon08:08:331
98499965,1cyclictest16956-21cat08:45:161
98499965,1cyclictest12510-21cpuspeed_turbos09:30:131
98499964,2cyclictest771-21snmpd09:47:341
98499963,2cyclictest27304-21ssh10:56:471
98499962,3cyclictest7833-21ssh11:10:221
98499962,3cyclictest20484-21ssh10:50:111
98499962,3cyclictest11947-21ssh09:29:401
98439966,0cyclictest771-21snmpd10:48:290
98439966,0cyclictest771-21snmpd10:31:280
98439966,0cyclictest771-21snmpd08:27:480
98439966,0cyclictest769-21lldpd08:44:500
98439966,0cyclictest524-21avahi-daemon08:54:530
98439965,1cyclictest524-21avahi-daemon11:21:300
98439965,1cyclictest1926-21cpuspeed_turbos11:05:150
98439964,2cyclictest771-21snmpd11:48:010
98439964,2cyclictest771-21snmpd09:16:450
98439963,3cyclictest771-21snmpd11:58:010
98439962,4cyclictest550-21NetworkManager07:10:150
98619955,0cyclictest771-21snmpd07:45:013
98619955,0cyclictest771-21snmpd07:15:493
98619954,1cyclictest771-21snmpd09:26:423
98619954,1cyclictest771-21snmpd08:37:543
98619953,2cyclictest771-21snmpd11:25:093
98619953,2cyclictest749-21ntpd12:10:203
98619953,2cyclictest749-21ntpd08:45:003
98619952,3cyclictest550-21NetworkManager08:08:213
98619952,3cyclictest20520-21head07:35:153
98619952,2cyclictest8514-21rm11:46:233
98619952,2cyclictest8432-21dpkg08:25:143
98619952,2cyclictest5615-21grep08:15:213
98619952,2cyclictest14768-21sh10:07:253
98619952,2cyclictest14768-21sh10:07:243
98619952,2cyclictest11590-21cat11:50:153
98619951,4cyclictest550-21NetworkManager07:30:523
98619950,5cyclictest3207-21ntpq08:10:183
98619950,3cyclictest25533-21awk09:45:013
98619950,2cyclictest2667450ssh11:31:233
98499954,1cyclictest771-21snmpd07:24:251
98499954,1cyclictest23780-21ntpq09:00:171
98499953,2cyclictest771-21snmpd08:57:041
98499953,2cyclictest255-20systemd-journal10:45:551
98499952,2cyclictest30817-21rm11:35:311
98499952,2cyclictest21839-21seq07:38:501
98499952,2cyclictest21663-21cat12:00:201
98499952,2cyclictest19061-21sh10:11:571
98499952,2cyclictest13117-21ssh11:16:391
98499952,2cyclictest11837-21sshd08:30:331
98499952,2cyclictest11577-21ssh10:04:391
98499951,3cyclictest2133-21sed12:15:151
98439955,0cyclictest524-21avahi-daemon09:21:160
98439955,0cyclictest524-21avahi-daemon07:29:260
98439955,0cyclictest11746-21gltestperf12:25:160
98439953,2cyclictest771-21snmpd08:06:050
98439953,2cyclictest771-21snmpd07:15:030
98439953,2cyclictest0-21swapper/012:10:120
98439952,2cyclictest586-21rs:main0
98439952,2cyclictest5036-21ssh09:56:160
98439952,2cyclictest28211-21ssh12:08:200
98439952,2cyclictest11901-21pluginstate11:15:170
98439951,3cyclictest7834-21pmu-power10:35:200
98439951,3cyclictest27573-21ssh09:45:430
98439951,3cyclictest24049-21ssh12:04:080
98439950,4cyclictest2395-21ntpq11:40:170
98619942,2cyclictest6847-21cat08:20:133
98619942,2cyclictest550-21NetworkManager07:50:123
98619942,2cyclictest16545-21cat07:25:133
98619941,2cyclictest25038-21cpu09:05:133
98499944,0cyclictest524-21avahi-daemon08:35:031
98499941,2cyclictest25612-21missed_timers09:05:161
98499940,3cyclictest18935-21/usr/sbin/munin08:50:111
98439942,2cyclictest8393-21cat09:25:150
98439940,4cyclictest16950-21ntpq12:30:170
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional