You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-12-04 - 18:26

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #8, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack8slot6s.osadl.org (updated Thu Dec 04, 2025 12:43:43)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
5016992019,1cyclictest66850irq/125-lan11:55:121
5016991918,1cyclictest66850irq/125-lan12:20:141
5016991918,1cyclictest66850irq/125-lan10:55:151
5016991918,1cyclictest66850irq/125-lan10:21:391
5016991918,1cyclictest66850irq/125-lan09:43:351
5016991918,1cyclictest66850irq/125-lan09:13:421
5016991818,0cyclictest66850irq/125-lan12:26:581
5016991818,0cyclictest66850irq/125-lan11:10:231
5016991818,0cyclictest66850irq/125-lan11:05:191
5016991818,0cyclictest66850irq/125-lan10:48:291
5016991817,1cyclictest66850irq/125-lan12:02:161
5016991817,1cyclictest66850irq/125-lan11:30:151
5016991817,1cyclictest66850irq/125-lan11:00:031
5016991817,1cyclictest66850irq/125-lan10:30:151
5016991817,1cyclictest66850irq/125-lan10:25:131
5016991817,1cyclictest66850irq/125-lan10:10:161
5016991817,1cyclictest66850irq/125-lan09:48:271
5016991817,1cyclictest66850irq/125-lan08:44:591
5016991817,1cyclictest66850irq/125-lan08:39:231
5016991817,1cyclictest66850irq/125-lan08:33:321
5016991717,0cyclictest66850irq/125-lan12:30:141
5016991717,0cyclictest66850irq/125-lan12:12:101
5016991717,0cyclictest66850irq/125-lan12:08:441
5016991717,0cyclictest66850irq/125-lan10:42:391
5016991716,1cyclictest66850irq/125-lan12:35:221
5016991716,1cyclictest66850irq/125-lan11:40:141
5016991716,1cyclictest66850irq/125-lan11:20:161
5016991716,1cyclictest66850irq/125-lan11:17:091
5016991716,1cyclictest66850irq/125-lan10:35:551
5016991716,1cyclictest66850irq/125-lan09:55:171
5016991716,1cyclictest66850irq/125-lan09:54:571
5016991716,1cyclictest66850irq/125-lan09:36:551
5016991716,1cyclictest66850irq/125-lan09:30:191
5016991716,1cyclictest66850irq/125-lan09:17:241
5016991716,1cyclictest66850irq/125-lan08:09:491
5016991716,1cyclictest66850irq/125-lan07:20:171
5016991716,1cyclictest66850irq/125-lan07:10:111
5016991616,0cyclictest66850irq/125-lan12:15:181
5016991616,0cyclictest66850irq/125-lan11:35:471
5016991616,0cyclictest66850irq/125-lan11:29:251
5016991616,0cyclictest66850irq/125-lan09:05:211
5016991616,0cyclictest66850irq/125-lan07:45:141
5016991615,1cyclictest66850irq/125-lan11:45:011
5016991615,1cyclictest66850irq/125-lan10:50:141
5016991615,1cyclictest66850irq/125-lan10:18:211
5016991615,1cyclictest66850irq/125-lan10:05:291
5016991615,1cyclictest66850irq/125-lan10:03:321
5016991515,0cyclictest66850irq/125-lan09:21:051
5016991515,0cyclictest66850irq/125-lan09:01:051
5016991514,1cyclictest66850irq/125-lan11:50:041
5016991514,1cyclictest66850irq/125-lan07:55:151
5016991414,0cyclictest66850irq/125-lan08:25:041
5016991414,0cyclictest66850irq/125-lan08:20:121
5016991414,0cyclictest66850irq/125-lan08:11:221
5016991414,0cyclictest66850irq/125-lan07:30:131
5016991413,1cyclictest66850irq/125-lan08:45:151
5016991413,1cyclictest66850irq/125-lan08:15:171
5016991313,0cyclictest66850irq/125-lan08:55:161
5016991212,0cyclictest66850irq/125-lan07:15:151
5016991211,1cyclictest66850irq/125-lan08:50:441
5016991211,1cyclictest66850irq/125-lan07:35:121
5016991111,0cyclictest66850irq/125-lan07:25:151
502099109,1cyclictest524-21avahi-daemon11:48:192
502099108,1cyclictest771-21snmpd11:42:302
5016991010,0cyclictest66850irq/125-lan07:43:211
5014991010,0cyclictest771-21snmpd07:25:170
50249998,1cyclictest771-21snmpd10:05:083
50249998,1cyclictest771-21snmpd09:29:333
50249998,1cyclictest769-21lldpd09:56:423
50249998,1cyclictest769-21lldpd09:43:163
50249996,3cyclictest27270-21cat09:15:153
50209998,1cyclictest771-21snmpd08:30:532
50209998,1cyclictest771-21snmpd08:12:072
50209998,1cyclictest769-21lldpd08:50:012
50209996,2cyclictest771-21snmpd12:28:522
50169998,1cyclictest66850irq/125-lan08:00:131
50169998,1cyclictest66850irq/125-lan07:50:131
50149999,0cyclictest771-21snmpd08:32:260
50149999,0cyclictest771-21snmpd07:40:090
50149999,0cyclictest6908-21cat12:25:160
50149999,0cyclictest28340-21cat08:05:160
50149999,0cyclictest14911-21cat10:15:140
50149998,1cyclictest769-21lldpd07:37:510
50149996,3cyclictest9880-21cat11:55:140
50249988,0cyclictest769-21lldpd11:26:453
50249988,0cyclictest769-21lldpd09:51:433
50249988,0cyclictest769-21lldpd08:18:123
50249987,1cyclictest771-21snmpd12:24:313
50249987,1cyclictest771-21snmpd10:39:303
50249987,1cyclictest771-21snmpd07:48:283
50249987,1cyclictest771-21snmpd07:31:453
50249987,1cyclictest769-21lldpd07:27:323
50249987,1cyclictest517-21systemd-resolve11:35:103
50249987,1cyclictest26145-21fschecks_time08:00:123
50249986,2cyclictest771-21snmpd10:25:023
50209988,0cyclictest771-21snmpd12:30:212
50209988,0cyclictest771-21snmpd11:31:222
50209988,0cyclictest771-21snmpd11:21:292
50209988,0cyclictest771-21snmpd10:54:342
50209988,0cyclictest771-21snmpd10:44:042
50209988,0cyclictest771-21snmpd10:09:232
50209988,0cyclictest771-21snmpd10:03:112
50209988,0cyclictest769-21lldpd07:20:502
50209987,1cyclictest771-21snmpd09:36:222
50209987,1cyclictest771-21snmpd09:26:422
50209987,1cyclictest524-21avahi-daemon09:16:062
50209987,1cyclictest14019-21ntp_kernel_err11:25:172
50209986,2cyclictest771-21snmpd09:04:362
50149988,0cyclictest771-21snmpd11:44:030
50149988,0cyclictest771-21snmpd11:31:550
50149988,0cyclictest771-21snmpd11:18:400
50149988,0cyclictest771-21snmpd11:13:500
50149988,0cyclictest771-21snmpd10:38:450
50149988,0cyclictest771-21snmpd10:30:590
50149988,0cyclictest771-21snmpd10:24:360
50149988,0cyclictest771-21snmpd09:57:070
50149988,0cyclictest771-21snmpd09:32:430
50149988,0cyclictest771-21snmpd09:18:190
50149988,0cyclictest771-21snmpd08:53:360
50149988,0cyclictest771-21snmpd08:22:290
50149988,0cyclictest771-21snmpd08:19:140
50149988,0cyclictest771-21snmpd08:10:460
50149988,0cyclictest771-21snmpd08:00:160
50149988,0cyclictest769-21lldpd12:17:090
50149988,0cyclictest769-21lldpd12:03:270
50149988,0cyclictest14555-21cat10:50:140
50149987,1cyclictest771-21snmpd09:26:060
50149987,1cyclictest769-21lldpd07:59:070
50149987,1cyclictest524-21avahi-daemon12:30:180
50149987,1cyclictest524-21avahi-daemon11:29:520
50149987,1cyclictest517-21systemd-resolve09:54:500
50149986,2cyclictest771-21snmpd08:59:260
50149986,2cyclictest749-21ntpd12:20:060
50149986,2cyclictest749-21ntpd10:05:180
50249977,0cyclictest771-21snmpd10:15:573
50249977,0cyclictest769-21lldpd12:13:193
50249977,0cyclictest769-21lldpd11:43:073
50249977,0cyclictest769-21lldpd11:02:023
50249977,0cyclictest769-21lldpd10:47:043
50249977,0cyclictest769-21lldpd10:33:063
50249977,0cyclictest769-21lldpd08:34:123
50249976,1cyclictest771-21snmpd09:05:263
50249976,1cyclictest524-21avahi-daemon10:23:293
50249975,2cyclictest771-21snmpd08:49:393
50249974,3cyclictest550-21NetworkManager12:09:183
50249973,3cyclictest586-21rs:main3
50209977,0cyclictest771-21snmpd12:19:542
50209977,0cyclictest771-21snmpd12:14:362
50209977,0cyclictest771-21snmpd12:02:292
50209977,0cyclictest771-21snmpd07:49:342
50209977,0cyclictest771-21snmpd07:19:362
50209977,0cyclictest769-21lldpd11:05:222
50209977,0cyclictest769-21lldpd07:54:452
50209976,1cyclictest769-21lldpd11:10:542
50209975,2cyclictest771-21snmpd11:59:362
50209975,2cyclictest771-21snmpd11:52:362
50209975,2cyclictest771-21snmpd11:01:572
50209975,2cyclictest771-21snmpd10:34:562
50209975,2cyclictest5636-21cat07:10:132
50209973,3cyclictest255-20systemd-journal10:17:152
50149977,0cyclictest771-21snmpd12:11:060
50149977,0cyclictest771-21snmpd10:42:400
50149977,0cyclictest771-21snmpd07:53:040
50149977,0cyclictest771-21snmpd07:15:470
50149977,0cyclictest769-21lldpd09:10:580
50149977,0cyclictest524-21avahi-daemon09:22:000
50149976,1cyclictest9656-21/usr/sbin/munin07:20:210
50149976,1cyclictest771-21snmpd11:54:210
50149976,1cyclictest771-21snmpd10:28:420
50149976,1cyclictest771-21snmpd09:38:460
50149976,1cyclictest771-21snmpd08:27:460
50149976,1cyclictest771-21snmpd07:14:560
50149976,1cyclictest749-21ntpd12:05:150
50149975,2cyclictest771-21snmpd11:37:480
50149975,2cyclictest771-21snmpd09:42:510
50149975,2cyclictest771-21snmpd09:01:090
50149975,2cyclictest771-21snmpd08:41:160
50249966,0cyclictest771-21snmpd11:34:473
50249966,0cyclictest771-21snmpd09:32:553
50249966,0cyclictest769-21lldpd11:56:463
50249966,0cyclictest769-21lldpd09:36:353
50249966,0cyclictest524-21avahi-daemon07:39:373
50249965,1cyclictest771-21snmpd07:16:063
50249965,1cyclictest769-21lldpd08:53:023
50249964,2cyclictest771-21snmpd08:38:163
50249964,2cyclictest771-21snmpd07:14:263
50249964,2cyclictest550-21NetworkManager10:55:423
50249964,2cyclictest29310-21ntpq12:15:163
50249964,2cyclictest12889-21ls11:25:003
50249963,3cyclictest550-21NetworkManager10:41:363
50249963,3cyclictest28183-21perl08:05:133
50249963,2cyclictest5609-21apt-get12:25:013
50249963,2cyclictest255-20systemd-journal10:51:453
50249962,3cyclictest9682-21dpkg07:20:133
50249962,3cyclictest16109-21ssh12:01:263
50249962,3cyclictest13461-21ssh12:31:263
50209966,0cyclictest771-21snmpd11:37:152
50209966,0cyclictest771-21snmpd07:58:242
50209966,0cyclictest769-21lldpd11:17:562
50209966,0cyclictest769-21lldpd10:13:192
50209966,0cyclictest769-21lldpd09:48:492
50209966,0cyclictest28257-21gltestperf08:05:142
50209965,1cyclictest771-21snmpd07:27:262
50209965,1cyclictest524-21avahi-daemon08:20:202
50209964,2cyclictest771-21snmpd08:45:062
50209964,2cyclictest771-21snmpd08:02:432
50209964,2cyclictest17907-21cut07:40:142
50209964,2cyclictest14129-21cat07:30:142
50209963,3cyclictest771-21snmpd08:59:412
50209963,3cyclictest550-21NetworkManager12:35:242
50209963,3cyclictest550-21NetworkManager10:56:172
50209963,3cyclictest18956-21ssh10:20:122
50209963,2cyclictest771-21snmpd09:50:022
50209963,2cyclictest2786-21apt09:25:002
50209962,3cyclictest255-20systemd-journal10:25:172
50209962,3cyclictest1777-21ntpq12:20:162
50209962,3cyclictest16667-21memory09:40:142
50209962,3cyclictest10226-21sed10:45:162
50209961,5cyclictest550-21NetworkManager09:31:122
50149966,0cyclictest771-21snmpd11:01:180
50149965,1cyclictest20589-21cat09:05:160
50149964,2cyclictest771-21snmpd11:21:320
50149963,3cyclictest29447-21ssh11:05:490
50149963,2cyclictest586-21rs:main0
50149963,2cyclictest13994-21ssh10:14:540
50149963,0cyclictest32442-21ssh11:45:120
50249955,0cyclictest769-21lldpd07:44:043
50249954,1cyclictest771-21snmpd11:12:593
50249954,1cyclictest771-21snmpd10:12:043
50249954,1cyclictest771-21snmpd09:46:143
50249954,1cyclictest524-21avahi-daemon11:15:133
50249954,1cyclictest25607-21ls07:59:593
50249953,2cyclictest771-21snmpd07:50:033
50249953,2cyclictest550-21NetworkManager09:10:273
50249953,2cyclictest10839-21ntpq08:40:183
50249952,3cyclictest29133-21sed11:05:233
50249952,2cyclictest6408-21ssh11:50:393
50249952,2cyclictest4714-21ld08:25:153
50249952,2cyclictest2751-21ld08:20:163
50249952,2cyclictest17852-21ssh12:35:353
50249950,4cyclictest1693-21ntpq10:00:163
50209955,0cyclictest771-21snmpd07:35:082
50209954,1cyclictest22527-21fschecks_count09:10:142
50209953,2cyclictest308-21sort10:35:152
50209953,2cyclictest19960-21sed12:05:222
50209952,2cyclictest609-21kworker/u16:3+nfsiod09:55:402
50209952,2cyclictest21519-21vmstat09:05:242
50209952,2cyclictest1131-21ls08:15:222
50209952,2cyclictest10349-21fschecks_time08:40:142
50209951,3cyclictest590-21gmain08:35:032
50149953,2cyclictest749-21ntpd10:55:330
50149953,2cyclictest749-21ntpd07:47:200
50149953,2cyclictest550-21NetworkManager10:45:300
50149953,2cyclictest541-21systemd-logind07:30:210
50149953,2cyclictest0-21swapper/010:01:020
50149953,2cyclictest0-21swapper/009:46:060
50249941,2cyclictest16539-21if_err_lan08:55:163
50249940,3cyclictest30220-21dpkg08:10:013
50249940,3cyclictest18313-21/usr/sbin/munin09:00:133
50209940,3cyclictest4299-21fw_forwarded_lo08:25:122
50149942,2cyclictest749-21ntpd08:45:110
50149941,2cyclictest8296-21cut08:35:130
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional