You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-03 - 19:36

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #8, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack8slot6s.osadl.org (updated Mon Nov 03, 2025 12:43:44)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
24308992019,1cyclictest66850irq/125-lan12:13:122
24308991918,1cyclictest66850irq/125-lan12:35:182
24308991918,1cyclictest66850irq/125-lan10:15:442
24308991918,1cyclictest66850irq/125-lan10:13:022
24308991918,1cyclictest66850irq/125-lan09:48:552
24308991818,0cyclictest66850irq/125-lan12:24:042
24308991817,1cyclictest66850irq/125-lan12:05:182
24308991817,1cyclictest66850irq/125-lan12:01:142
24308991817,1cyclictest66850irq/125-lan11:40:192
24308991817,1cyclictest66850irq/125-lan11:26:012
24308991817,1cyclictest66850irq/125-lan11:15:122
24308991817,1cyclictest66850irq/125-lan11:11:232
24308991817,1cyclictest66850irq/125-lan11:04:202
24308991817,1cyclictest66850irq/125-lan10:50:112
24308991817,1cyclictest66850irq/125-lan10:47:392
24308991817,1cyclictest66850irq/125-lan10:43:462
24308991817,1cyclictest66850irq/125-lan10:30:182
24308991817,1cyclictest66850irq/125-lan09:59:052
24308991817,1cyclictest66850irq/125-lan09:42:302
24308991817,1cyclictest66850irq/125-lan09:28:072
24308991817,1cyclictest66850irq/125-lan09:15:192
24308991817,1cyclictest66850irq/125-lan07:50:132
24308991817,1cyclictest66850irq/125-lan07:40:222
24308991817,1cyclictest66850irq/125-lan07:22:052
24308991717,0cyclictest66850irq/125-lan11:30:222
24308991717,0cyclictest66850irq/125-lan10:36:032
24308991717,0cyclictest66850irq/125-lan10:05:542
24308991717,0cyclictest66850irq/125-lan08:50:152
24308991717,0cyclictest66850irq/125-lan08:25:232
24308991717,0cyclictest66850irq/125-lan08:05:192
24308991717,0cyclictest66850irq/125-lan07:25:112
24308991716,1cyclictest66850irq/125-lan12:31:182
24308991716,1cyclictest66850irq/125-lan11:58:442
24308991716,1cyclictest66850irq/125-lan11:52:182
24308991716,1cyclictest66850irq/125-lan11:48:422
24308991716,1cyclictest66850irq/125-lan11:36:092
24308991716,1cyclictest66850irq/125-lan10:25:152
24308991716,1cyclictest66850irq/125-lan10:02:242
24308991716,1cyclictest66850irq/125-lan09:54:522
24308991716,1cyclictest66850irq/125-lan09:39:332
24308991716,1cyclictest66850irq/125-lan09:10:142
24308991616,0cyclictest66850irq/125-lan12:16:092
24308991616,0cyclictest66850irq/125-lan09:22:282
24308991616,0cyclictest66850irq/125-lan08:30:152
24308991616,0cyclictest66850irq/125-lan08:10:162
24308991615,1cyclictest66850irq/125-lan12:26:102
24308991615,1cyclictest66850irq/125-lan11:22:252
24308991615,1cyclictest66850irq/125-lan10:57:142
24308991615,1cyclictest66850irq/125-lan10:20:152
24308991615,1cyclictest66850irq/125-lan08:45:132
24308991615,1cyclictest66850irq/125-lan07:30:212
24308991615,1cyclictest66850irq/125-lan07:10:162
24308991515,0cyclictest66850irq/125-lan11:05:162
24308991514,1cyclictest66850irq/125-lan09:31:152
24308991514,1cyclictest66850irq/125-lan09:00:162
24308991514,1cyclictest66850irq/125-lan08:17:412
24308991514,1cyclictest66850irq/125-lan07:35:132
24308991414,0cyclictest66850irq/125-lan09:05:162
24308991414,0cyclictest66850irq/125-lan08:55:322
24308991414,0cyclictest66850irq/125-lan08:20:132
24308991414,0cyclictest66850irq/125-lan07:55:172
24308991414,0cyclictest66850irq/125-lan07:15:122
24308991413,1cyclictest66850irq/125-lan07:45:162
24308991313,0cyclictest66850irq/125-lan08:00:182
24308991313,0cyclictest66850irq/125-lan08:00:172
2431399114,4cyclictest6045-21cat10:20:143
2431399111,6cyclictest29445-21cat09:35:003
24308991111,0cyclictest66850irq/125-lan08:40:182
24300991110,1cyclictest771-21snmpd12:14:550
2431399108,2cyclictest771-21snmpd10:52:373
2431399108,2cyclictest771-21snmpd08:24:473
2431399108,2cyclictest517-21systemd-resolve11:00:043
2431399106,3cyclictest524-21avahi-daemon09:37:233
2431399100,2cyclictest21579-21sed09:25:003
243139998,1cyclictest771-21snmpd08:47:273
243139998,1cyclictest769-21lldpd10:41:023
243139998,1cyclictest769-21lldpd08:50:493
243139998,1cyclictest517-21systemd-resolve09:14:283
243139998,1cyclictest13551-21ntp_offset11:05:193
243139996,3cyclictest771-21snmpd07:33:243
243139995,4cyclictest771-21snmpd07:49:143
243139993,5cyclictest16003-21sed09:55:143
243139990,6cyclictest10727-21ssh09:50:003
243139990,6cyclictest10039-21diskmemload10:17:143
243139990,3cyclictest17536-21rm10:33:383
243139990,2cyclictest31565-21ssh12:03:333
243139990,2cyclictest28573-21/usr/sbin/munin12:35:163
243139990,2cyclictest27958-21ssh11:22:023
243059998,1cyclictest771-21snmpd08:16:161
243009998,1cyclictest771-21snmpd11:08:230
243009998,1cyclictest771-21snmpd10:01:100
243139988,0cyclictest771-21snmpd09:49:263
243139988,0cyclictest771-21snmpd09:08:103
243139986,2cyclictest771-21snmpd12:08:553
243139986,2cyclictest771-21snmpd11:46:213
243139986,2cyclictest771-21snmpd08:56:383
243139986,2cyclictest771-21snmpd08:02:403
243139986,2cyclictest771-21snmpd08:02:403
243139986,2cyclictest771-21snmpd07:35:573
243139986,2cyclictest769-21lldpd10:05:383
243139986,2cyclictest749-21ntpd08:15:243
243139986,2cyclictest2478-21cat11:30:163
243139985,3cyclictest771-21snmpd11:16:023
243139984,3cyclictest28155-21apt-get08:35:143
243139983,4cyclictest550-21NetworkManager07:50:233
243139982,6cyclictest12073-21awk07:55:233
243139980,3cyclictest31081-21sendmail_mailqu11:25:213
243139980,3cyclictest29675-21ssh10:46:503
243139980,3cyclictest26349-21chrt12:33:113
243139980,2cyclictest23500-21ssh09:26:173
243139980,2cyclictest18795-21ssh11:11:323
243139980,2cyclictest16882-21ssh09:18:593
243139980,2cyclictest1598-21sh09:40:053
243139980,2cyclictest12547-21ssh12:17:433
243089987,1cyclictest66850irq/125-lan08:38:212
243059988,0cyclictest771-21snmpd09:36:231
243059988,0cyclictest771-21snmpd08:14:121
243059988,0cyclictest769-21lldpd11:35:321
243059988,0cyclictest769-21lldpd10:27:391
243059988,0cyclictest769-21lldpd09:54:421
243059988,0cyclictest769-21lldpd09:22:081
243059988,0cyclictest769-21lldpd08:27:461
243059988,0cyclictest769-21lldpd08:20:531
243059988,0cyclictest769-21lldpd07:42:241
243059987,1cyclictest771-21snmpd12:32:351
243059987,1cyclictest771-21snmpd10:51:101
243059987,1cyclictest517-21systemd-resolve11:19:501
243059986,2cyclictest19684-21cat11:50:161
243059986,2cyclictest15843-21cat09:55:141
243009988,0cyclictest771-21snmpd10:22:120
243009988,0cyclictest771-21snmpd07:47:110
243009988,0cyclictest769-21lldpd12:09:200
243009988,0cyclictest769-21lldpd11:24:570
243009988,0cyclictest769-21lldpd10:19:250
243009988,0cyclictest769-21lldpd08:23:430
243009988,0cyclictest769-21lldpd08:19:270
243009987,1cyclictest771-21snmpd10:10:150
243009987,1cyclictest771-21snmpd09:13:130
243009986,2cyclictest771-21snmpd09:17:490
243009986,2cyclictest771-21snmpd08:53:510
243009986,2cyclictest771-21snmpd08:45:240
243009986,2cyclictest771-21snmpd07:52:420
243139975,2cyclictest771-21snmpd12:20:193
243139974,2cyclictest6157-21apt-get11:35:133
243139973,4cyclictest19888-21cpuspeed_turbos10:00:143
243139972,5cyclictest4494-21ntpq10:55:183
243139970,4cyclictest27331-21ntpq07:15:163
243139970,4cyclictest20095-21ntpq11:50:193
243139970,4cyclictest19551-21ntpq10:35:183
243139970,4cyclictest0-21swapper/307:20:143
243139970,3cyclictest5413-21ntpq07:40:173
243139970,3cyclictest31759-21ntpq07:25:143
243139970,3cyclictest30351-21sed10:10:203
243139970,3cyclictest25348-21ntp_states07:10:193
243139970,3cyclictest24895-21/usr/sbin/munin11:55:203
243139970,3cyclictest16727-21cat08:10:013
243139970,3cyclictest11214-21ld11:40:163
243139970,3cyclictest11071-21ntpq10:25:193
243139970,2cyclictest30754-21cut08:40:143
243139970,2cyclictest10039-21diskmemload12:10:193
243139970,1cyclictest186272sleep308:11:033
243059977,0cyclictest771-21snmpd10:37:461
243059977,0cyclictest771-21snmpd10:01:511
243059977,0cyclictest771-21snmpd09:09:191
243059977,0cyclictest771-21snmpd07:25:151
243059977,0cyclictest771-21snmpd07:23:441
243059977,0cyclictest769-21lldpd10:30:131
243059977,0cyclictest769-21lldpd09:01:091
243059977,0cyclictest769-21lldpd08:09:571
243059977,0cyclictest517-21systemd-resolve09:49:411
243059977,0cyclictest0-21swapper/111:40:191
243059976,1cyclictest517-21systemd-resolve11:46:091
243059976,1cyclictest2357-21head11:30:151
243059976,1cyclictest19168-21cat12:25:151
243059975,2cyclictest771-21snmpd08:59:171
243059975,2cyclictest771-21snmpd07:30:411
243059973,3cyclictest7606-21ssh12:11:541
243059973,3cyclictest255-20systemd-journal07:46:481
243059973,3cyclictest1187-21ssh10:15:031
243009977,0cyclictest771-21snmpd12:27:470
243009977,0cyclictest771-21snmpd12:20:400
243009977,0cyclictest769-21lldpd10:32:050
243009977,0cyclictest769-21lldpd09:31:340
243009977,0cyclictest769-21lldpd08:33:250
243009976,1cyclictest771-21snmpd08:55:120
243009976,1cyclictest24699-21gltestperf10:05:150
243009975,2cyclictest771-21snmpd09:48:350
243009975,2cyclictest771-21snmpd09:05:550
243009975,2cyclictest16798-21unixbench_singl09:55:210
243009974,2cyclictest771-21snmpd08:35:030
243009974,2cyclictest255-20systemd-journal12:32:530
243009973,3cyclictest15060-21fw_forwarded_lo08:05:150
243009972,5cyclictest550-21NetworkManager11:41:250
243139965,1cyclictest24479-21gltestperf08:25:133
243139960,3cyclictest6448-21df_inode09:00:143
243139960,3cyclictest26111-21seq08:30:113
243139960,2cyclictest19258-21cat12:25:153
243059966,0cyclictest769-21lldpd09:26:041
243059966,0cyclictest769-21lldpd08:52:211
243059966,0cyclictest524-21avahi-daemon08:48:181
243059966,0cyclictest524-21avahi-daemon08:01:281
243059966,0cyclictest524-21avahi-daemon08:01:271
243059965,1cyclictest771-21snmpd11:25:381
243059965,1cyclictest6452-21ntpq10:20:171
243059965,1cyclictest524-21avahi-daemon11:00:401
243059965,1cyclictest524-21avahi-daemon09:31:171
243059965,1cyclictest524-21avahi-daemon09:10:141
243059965,1cyclictest26215-21cpuspeed_turbos08:30:151
243059964,2cyclictest771-21snmpd11:09:381
243059964,2cyclictest26931-21cat07:15:131
243059964,2cyclictest255-20systemd-journal09:43:251
243059963,3cyclictest3964-21fw_packets10:55:151
243059963,2cyclictest32143-21ssh12:04:281
243059963,2cyclictest29056-21ssh10:45:431
243059963,2cyclictest255-20systemd-journal10:43:091
243059962,3cyclictest28981-21ssh12:35:201
243059962,3cyclictest25515-21cat10:05:221
243059962,3cyclictest17550-21ssh12:23:351
243059962,3cyclictest12653-21cut08:00:011
243009966,0cyclictest771-21snmpd07:41:450
243009966,0cyclictest769-21lldpd10:45:170
243009966,0cyclictest769-21lldpd10:44:410
243009966,0cyclictest769-21lldpd08:14:360
243009965,1cyclictest524-21avahi-daemon09:22:450
243009965,1cyclictest29040-21cpuspeed_turbos07:20:140
243009965,1cyclictest10386-21fschecks_count10:25:160
243009964,2cyclictest771-21snmpd12:16:550
243009962,3cyclictest901-21ssh09:39:130
243009962,3cyclictest31206-21ssh12:37:390
243009962,3cyclictest255-20systemd-journal10:51:560
243009962,3cyclictest1884-21sed07:30:220
243059954,1cyclictest524-21avahi-daemon10:11:351
243059952,2cyclictest586-21rs:main1
243059952,2cyclictest586-21rs:main1
243059952,2cyclictest586-21rs:main1
243059952,2cyclictest30272-21apt-key08:40:001
243059952,2cyclictest25457-21cut07:10:201
243059952,2cyclictest24357-21gcc11:55:171
243059952,2cyclictest16843-21awk11:10:121
243059952,2cyclictest16592-21ssh09:18:221
243059952,2cyclictest12350-21ssh12:17:251
243059950,3cyclictest3362-21ntpq07:35:171
243009955,0cyclictest771-21snmpd11:55:060
243009954,1cyclictest769-21lldpd07:58:250
243009954,1cyclictest517-21systemd-resolve11:50:430
243009953,2cyclictest771-21snmpd07:36:210
243009953,2cyclictest7665-21sshd09:00:360
243009953,2cyclictest19686-21ssh10:35:200
243009953,2cyclictest10039-21diskmemload09:44:090
243009952,3cyclictest550-21NetworkManager11:31:530
243009952,2cyclictest8899-21cut11:00:200
243009952,2cyclictest8463-21rm11:37:250
243009952,2cyclictest4648-21awk10:55:200
243009952,2cyclictest30148-21ssh12:01:130
243009952,2cyclictest255-20systemd-journal09:50:350
243009952,2cyclictest255-20systemd-journal07:26:350
243009952,2cyclictest25428-21awk08:25:260
243009952,2cyclictest25374-21smartctl07:10:190
243009952,2cyclictest23471-21ssh11:17:100
243009952,2cyclictest23437-21ssh09:26:110
243009952,2cyclictest15630-21netstat11:45:160
243059941,2cyclictest706-21fschecks_time12:05:151
243009941,2cyclictest31417-21ld08:40:200
243009941,2cyclictest29974-21ssh11:25:120
243009941,2cyclictest27214-21ld07:15:150
243009941,2cyclictest15878-21kworker/u16:2+nfsiod11:10:120
243009941,2cyclictest12848-21cut08:00:130
243009941,2cyclictest12848-21cut08:00:120
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional