You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-18 - 11:19

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #8, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack8slot6s.osadl.org (updated Tue Nov 18, 2025 00:43:43)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
29182991919,0cyclictest66850irq/125-lan21:25:163
29182991918,1cyclictest66850irq/125-lan23:55:173
29182991918,1cyclictest66850irq/125-lan23:21:233
29182991918,1cyclictest66850irq/125-lan23:04:373
29182991918,1cyclictest66850irq/125-lan22:31:423
29182991918,1cyclictest66850irq/125-lan22:12:463
29182991918,1cyclictest66850irq/125-lan21:30:473
29182991918,1cyclictest66850irq/125-lan00:32:163
29182991918,1cyclictest66850irq/125-lan00:05:213
29182991918,1cyclictest66850irq/125-lan00:03:563
29182991818,0cyclictest66850irq/125-lan23:37:033
29182991818,0cyclictest66850irq/125-lan23:25:223
29182991818,0cyclictest66850irq/125-lan22:48:043
29182991818,0cyclictest66850irq/125-lan22:27:003
29182991818,0cyclictest66850irq/125-lan21:15:483
29182991818,0cyclictest66850irq/125-lan00:10:203
29182991817,1cyclictest66850irq/125-lan23:10:133
29182991817,1cyclictest66850irq/125-lan23:08:183
29182991817,1cyclictest66850irq/125-lan22:05:443
29182991817,1cyclictest66850irq/125-lan21:45:153
29182991817,1cyclictest66850irq/125-lan21:38:143
29182991817,1cyclictest66850irq/125-lan00:35:173
29182991817,1cyclictest66850irq/125-lan00:24:133
29182991717,0cyclictest66850irq/125-lan23:40:163
29182991717,0cyclictest66850irq/125-lan23:30:023
29182991717,0cyclictest66850irq/125-lan22:52:293
29182991717,0cyclictest66850irq/125-lan22:40:053
29182991717,0cyclictest66850irq/125-lan22:22:143
29182991717,0cyclictest66850irq/125-lan22:00:183
29182991717,0cyclictest66850irq/125-lan21:20:083
29182991716,1cyclictest66850irq/125-lan23:50:193
29182991716,1cyclictest66850irq/125-lan23:15:363
29182991716,1cyclictest66850irq/125-lan22:55:183
29182991716,1cyclictest66850irq/125-lan22:35:203
29182991716,1cyclictest66850irq/125-lan21:55:233
29182991716,1cyclictest66850irq/125-lan20:50:133
29182991716,1cyclictest66850irq/125-lan20:10:003
29182991616,0cyclictest66850irq/125-lan22:15:183
29182991616,0cyclictest66850irq/125-lan21:52:433
29182991616,0cyclictest66850irq/125-lan20:02:433
29182991616,0cyclictest66850irq/125-lan19:35:193
29182991616,0cyclictest66850irq/125-lan00:26:123
29182991615,1cyclictest66850irq/125-lan23:47:373
29182991615,1cyclictest66850irq/125-lan21:40:153
29182991615,1cyclictest66850irq/125-lan21:10:213
29182991615,1cyclictest66850irq/125-lan20:55:183
29182991615,1cyclictest66850irq/125-lan20:30:213
29182991615,1cyclictest66850irq/125-lan20:25:233
29182991615,1cyclictest66850irq/125-lan19:20:203
29182991515,0cyclictest66850irq/125-lan19:10:153
29182991514,1cyclictest66850irq/125-lan21:05:133
29182991514,1cyclictest66850irq/125-lan20:43:583
29182991514,1cyclictest66850irq/125-lan20:35:203
29182991514,1cyclictest66850irq/125-lan20:20:133
29182991514,1cyclictest66850irq/125-lan20:15:153
29182991514,1cyclictest66850irq/125-lan19:50:133
29182991514,1cyclictest66850irq/125-lan00:15:183
29182991414,0cyclictest66850irq/125-lan20:45:143
29182991312,1cyclictest66850irq/125-lan20:10:013
29182991312,1cyclictest66850irq/125-lan19:40:163
29182991211,1cyclictest66850irq/125-lan19:45:243
29182991111,0cyclictest66850irq/125-lan19:30:193
29182991111,0cyclictest66850irq/125-lan19:25:173
29182991110,1cyclictest66850irq/125-lan21:00:193
29182991110,1cyclictest66850irq/125-lan19:55:153
2917899119,2cyclictest749-21ntpd23:04:052
29182991010,0cyclictest66850irq/125-lan19:15:133
2917899109,1cyclictest749-21ntpd23:22:012
291789998,1cyclictest769-21lldpd00:31:142
291749998,1cyclictest771-21snmpd22:39:121
291749998,1cyclictest771-21snmpd19:23:271
291749998,1cyclictest771-21snmpd00:10:511
291749998,1cyclictest769-21lldpd21:52:071
291749998,1cyclictest1670-21munin-plugin-st00:10:001
291699998,1cyclictest771-21snmpd20:38:060
291699998,1cyclictest771-21snmpd20:13:170
291699998,1cyclictest771-21snmpd00:12:450
291789988,0cyclictest771-21snmpd23:47:062
291789988,0cyclictest771-21snmpd19:27:042
291789988,0cyclictest769-21lldpd23:56:422
291789988,0cyclictest769-21lldpd20:19:452
291789987,1cyclictest771-21snmpd22:51:192
291789987,1cyclictest771-21snmpd22:00:542
291789987,1cyclictest771-21snmpd20:43:042
291789987,1cyclictest769-21lldpd00:04:122
291789987,1cyclictest524-21avahi-daemon00:29:442
291789987,1cyclictest524-21avahi-daemon00:15:112
291789987,1cyclictest517-21systemd-resolve00:37:132
291789986,2cyclictest771-21snmpd22:30:492
291789986,2cyclictest771-21snmpd21:40:052
291789986,2cyclictest11547-21cat19:45:172
291749988,0cyclictest771-21snmpd23:38:481
291749988,0cyclictest771-21snmpd23:26:201
291749988,0cyclictest771-21snmpd23:17:321
291749988,0cyclictest771-21snmpd21:25:171
291749988,0cyclictest771-21snmpd20:55:071
291749988,0cyclictest769-21lldpd22:15:301
291749988,0cyclictest769-21lldpd20:41:191
291749988,0cyclictest769-21lldpd20:00:361
291749988,0cyclictest769-21lldpd19:14:191
291749988,0cyclictest769-21lldpd00:26:231
291749987,1cyclictest771-21snmpd20:36:201
291749987,1cyclictest771-21snmpd19:41:231
291749987,1cyclictest524-21avahi-daemon22:00:251
291749987,1cyclictest524-21avahi-daemon00:38:071
291749986,2cyclictest771-21snmpd19:17:001
291749986,2cyclictest771-21snmpd00:33:141
291749985,3cyclictest771-21snmpd21:23:371
291699988,0cyclictest771-21snmpd23:04:140
291699988,0cyclictest771-21snmpd21:22:010
291699988,0cyclictest771-21snmpd20:00:140
291699988,0cyclictest771-21snmpd00:06:430
291699988,0cyclictest769-21lldpd23:37:250
291699988,0cyclictest769-21lldpd00:04:320
291699987,1cyclictest771-21snmpd22:18:000
291699987,1cyclictest771-21snmpd21:54:290
291699987,1cyclictest771-21snmpd21:03:130
291699987,1cyclictest771-21snmpd20:19:360
291699987,1cyclictest771-21snmpd19:56:040
291699987,1cyclictest771-21snmpd19:12:060
291699987,1cyclictest769-21lldpd19:40:190
291699987,1cyclictest749-21ntpd21:10:190
291699987,1cyclictest524-21avahi-daemon22:39:060
291699987,1cyclictest517-21systemd-resolve00:20:330
291699986,2cyclictest771-21snmpd23:53:490
291699986,2cyclictest771-21snmpd22:24:450
291699986,2cyclictest771-21snmpd19:53:340
291699986,2cyclictest771-21snmpd19:23:210
291699986,2cyclictest771-21snmpd00:18:100
291699986,2cyclictest21069-21cat00:30:150
291789977,0cyclictest771-21snmpd23:36:592
291789977,0cyclictest771-21snmpd22:41:582
291789977,0cyclictest771-21snmpd22:13:242
291789977,0cyclictest771-21snmpd21:06:392
291789977,0cyclictest771-21snmpd19:21:182
291789977,0cyclictest769-21lldpd23:17:392
291789977,0cyclictest769-21lldpd23:05:022
291789977,0cyclictest769-21lldpd00:20:312
291789977,0cyclictest769-21lldpd00:12:492
291789976,1cyclictest771-21snmpd20:45:212
291789976,1cyclictest771-21snmpd20:10:542
291789976,1cyclictest771-21snmpd00:09:542
291789976,1cyclictest517-21systemd-resolve22:48:172
291789976,1cyclictest517-21systemd-resolve22:06:282
291789975,2cyclictest771-21snmpd21:03:572
291789974,3cyclictest550-21NetworkManager22:20:422
291749977,0cyclictest771-21snmpd23:42:171
291749977,0cyclictest771-21snmpd23:32:301
291749977,0cyclictest771-21snmpd22:09:471
291749977,0cyclictest771-21snmpd20:13:321
291749977,0cyclictest769-21lldpd23:59:251
291749977,0cyclictest769-21lldpd23:03:551
291749977,0cyclictest769-21lldpd21:33:411
291749977,0cyclictest769-21lldpd19:33:531
291749977,0cyclictest524-21avahi-daemon20:54:471
291749977,0cyclictest517-21systemd-resolve22:42:461
291749976,1cyclictest771-21snmpd21:00:491
291749976,1cyclictest769-21lldpd00:02:161
291749975,2cyclictest771-21snmpd23:07:551
291749975,2cyclictest771-21snmpd20:17:451
291749975,2cyclictest771-21snmpd19:46:201
291749974,3cyclictest255-20systemd-journal23:23:041
291749973,3cyclictest9693-21ssh21:36:141
291699977,0cyclictest771-21snmpd23:12:250
291699977,0cyclictest771-21snmpd21:59:410
291699977,0cyclictest771-21snmpd21:08:150
291699977,0cyclictest771-21snmpd20:29:020
291699977,0cyclictest769-21lldpd22:26:570
291699977,0cyclictest769-21lldpd22:07:010
291699977,0cyclictest769-21lldpd21:36:380
291699977,0cyclictest769-21lldpd20:05:370
291699977,0cyclictest769-21lldpd00:27:350
291699976,1cyclictest771-21snmpd22:52:080
291699976,1cyclictest769-21lldpd23:48:170
291699976,1cyclictest31431-21gltestperf22:30:160
291699975,2cyclictest771-21snmpd23:42:290
291699975,2cyclictest771-21snmpd23:16:170
291699975,2cyclictest771-21snmpd22:03:450
291699975,2cyclictest771-21snmpd21:47:330
291699975,2cyclictest771-21snmpd19:32:080
291789966,0cyclictest771-21snmpd22:16:362
291789966,0cyclictest771-21snmpd21:31:292
291789966,0cyclictest771-21snmpd21:18:522
291789966,0cyclictest771-21snmpd19:50:342
291789966,0cyclictest769-21lldpd20:20:072
291789966,0cyclictest769-21lldpd19:31:432
291789966,0cyclictest524-21avahi-daemon19:42:212
291789965,1cyclictest771-21snmpd21:39:192
291789965,1cyclictest771-21snmpd20:52:332
291789965,1cyclictest6256-21cpuspeed_turbos23:10:122
291789963,2cyclictest586-21rs:main2
291789963,2cyclictest31552-21ssh21:25:402
291789962,3cyclictest29300-21ssh22:28:042
291789962,3cyclictest23496-21ssh23:26:052
291789962,3cyclictest17645-21ssh21:12:582
291749966,0cyclictest771-21snmpd22:55:041
291749966,0cyclictest769-21lldpd21:15:251
291749966,0cyclictest524-21avahi-daemon20:08:141
291749965,1cyclictest771-21snmpd22:31:011
291749965,1cyclictest517-21systemd-resolve21:45:471
291749964,2cyclictest550-21NetworkManager21:57:401
291749964,2cyclictest4991-21cron19:30:011
291749963,3cyclictest14298-21apt-get23:50:161
291749963,2cyclictest8073-21ssh23:11:001
291749963,2cyclictest26964-21pmu-power22:25:201
291749963,2cyclictest255-20systemd-journal00:16:591
291749963,2cyclictest14491-21ssh21:40:481
291749962,4cyclictest26303-21tr20:20:151
291749962,3cyclictest9658-21sed23:45:151
291749962,3cyclictest19020-21cat22:50:131
291749962,3cyclictest15363-21ssh22:45:311
291749962,3cyclictest11579-21fw_forwarded_lo00:20:141
291699966,0cyclictest771-21snmpd21:30:410
291699966,0cyclictest771-21snmpd21:27:340
291699966,0cyclictest769-21lldpd23:32:370
291699966,0cyclictest769-21lldpd23:06:590
291699966,0cyclictest769-21lldpd22:12:310
291699966,0cyclictest7634-21gpgv20:55:000
291699965,1cyclictest769-21lldpd20:21:550
291699965,1cyclictest524-21avahi-daemon20:45:360
291699965,1cyclictest28313-21idleruntime-cro23:00:000
291699964,2cyclictest771-21snmpd22:48:490
291699964,2cyclictest771-21snmpd20:40:120
291699964,2cyclictest771-21snmpd20:34:450
291699964,2cyclictest771-21snmpd19:15:220
291699963,2cyclictest22113-21ssh21:16:490
291699963,2cyclictest12565-21ssh21:40:000
291699962,3cyclictest27385-21ssh00:36:320
291699962,3cyclictest23795-21ssh23:26:310
291699962,3cyclictest11059-21seq22:41:380
291699961,5cyclictest19440-21tr23:55:120
291699960,4cyclictest17446-21ntpq23:20:190
291789955,0cyclictest771-21snmpd19:38:552
291789955,0cyclictest769-21lldpd19:15:262
291789954,1cyclictest255-20systemd-journal19:55:092
291789954,1cyclictest0-21swapper/223:40:172
291789953,2cyclictest771-21snmpd20:55:462
291789953,2cyclictest749-21ntpd21:50:532
291789953,2cyclictest25276-21ssh22:55:292
291789952,3cyclictest550-21NetworkManager23:31:042
291789952,2cyclictest30454-21ssh21:57:002
291789952,2cyclictest26587-21ssh21:20:532
291789952,2cyclictest17845-21ssh23:53:292
291789950,4cyclictest31464-21munin-run20:35:002
291789950,4cyclictest20017-21ntpq20:05:192
291749955,0cyclictest769-21lldpd20:31:071
291749954,1cyclictest28153-21irqstats20:25:171
291749954,1cyclictest21101-21fschecks_count22:20:121
291749952,2cyclictest7500-21if_lan19:35:151
291749952,2cyclictest18404-21ssh21:14:141
291749952,2cyclictest16076-21grep19:55:221
291749952,2cyclictest14140-21sendmail_mailst19:50:211
291749951,3cyclictest3816-21cat20:45:161
291699955,0cyclictest769-21lldpd19:46:000
291699955,0cyclictest769-21lldpd19:36:450
291699954,1cyclictest8968-21ntp_192.168.11420:55:200
291699950,4cyclictest3797-21ntpq19:25:170
291789942,2cyclictest29958-21wc19:10:182
291789942,2cyclictest28325-21grep20:25:182
291789942,2cyclictest17176-21apt-get20:00:122
291789941,2cyclictest32283-21cat20:35:192
291749940,3cyclictest12604-21fw_conntrack21:05:141
291749940,3cyclictest10961-21fw_forwarded_lo22:10:131
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional