You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-10-30 - 20:13

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #8, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack8slot6s.osadl.org (updated Thu Oct 30, 2025 12:43:44)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
19414994545,0cyclictest25135-21fwupd08:42:263
19410991918,1cyclictest66850irq/125-lan11:26:372
19410991918,1cyclictest66850irq/125-lan10:26:322
19410991918,1cyclictest66850irq/125-lan10:26:312
19410991918,1cyclictest66850irq/125-lan10:19:172
19410991918,1cyclictest66850irq/125-lan07:30:172
19410991818,0cyclictest771-21snmpd12:00:412
19410991818,0cyclictest771-21snmpd11:56:572
19410991818,0cyclictest771-21snmpd10:33:352
19410991818,0cyclictest771-21snmpd08:46:092
19410991818,0cyclictest771-21snmpd08:05:592
19410991818,0cyclictest66850irq/125-lan10:51:472
19410991818,0cyclictest66850irq/125-lan10:40:162
19410991818,0cyclictest66850irq/125-lan09:47:052
19410991818,0cyclictest66850irq/125-lan09:27:332
19410991818,0cyclictest66850irq/125-lan09:20:402
19410991818,0cyclictest32549-21cat09:00:142
19410991818,0cyclictest16841-21cat11:15:132
19410991818,0cyclictest14458-21cat10:00:142
19410991817,1cyclictest66850irq/125-lan12:35:522
19410991817,1cyclictest66850irq/125-lan12:25:272
19410991817,1cyclictest66850irq/125-lan09:17:322
19410991817,1cyclictest66850irq/125-lan07:40:182
19410991717,0cyclictest771-21snmpd11:32:072
19410991717,0cyclictest66850irq/125-lan12:32:532
19410991717,0cyclictest66850irq/125-lan12:20:212
19410991717,0cyclictest66850irq/125-lan12:16:262
19410991717,0cyclictest66850irq/125-lan12:14:022
19410991717,0cyclictest66850irq/125-lan11:47:562
19410991717,0cyclictest66850irq/125-lan10:55:042
19410991717,0cyclictest66850irq/125-lan10:20:152
19410991717,0cyclictest66850irq/125-lan10:10:132
19410991717,0cyclictest66850irq/125-lan10:09:002
19410991717,0cyclictest66850irq/125-lan09:35:082
19410991717,0cyclictest66850irq/125-lan09:33:372
19410991717,0cyclictest66850irq/125-lan08:50:112
19410991717,0cyclictest66850irq/125-lan08:10:162
19410991717,0cyclictest66850irq/125-lan07:29:192
19410991716,1cyclictest66850irq/125-lan12:06:142
19410991716,1cyclictest66850irq/125-lan11:50:142
19410991716,1cyclictest66850irq/125-lan11:40:172
19410991716,1cyclictest66850irq/125-lan11:12:072
19410991716,1cyclictest66850irq/125-lan11:05:132
19410991716,1cyclictest66850irq/125-lan10:49:212
19410991716,1cyclictest66850irq/125-lan09:56:152
19410991716,1cyclictest66850irq/125-lan09:53:062
19410991716,1cyclictest66850irq/125-lan09:43:532
19410991716,1cyclictest66850irq/125-lan09:10:252
19410991716,1cyclictest66850irq/125-lan08:40:192
19410991616,0cyclictest66850irq/125-lan11:20:182
19410991616,0cyclictest66850irq/125-lan10:38:592
19410991616,0cyclictest66850irq/125-lan09:05:122
19410991616,0cyclictest66850irq/125-lan08:55:142
19410991616,0cyclictest66850irq/125-lan08:32:322
19410991616,0cyclictest66850irq/125-lan07:10:122
19410991615,1cyclictest66850irq/125-lan11:35:192
19410991615,1cyclictest66850irq/125-lan11:00:262
19410991414,0cyclictest66850irq/125-lan08:35:212
19410991414,0cyclictest66850irq/125-lan08:26:292
19410991414,0cyclictest66850irq/125-lan07:55:122
19410991414,0cyclictest66850irq/125-lan07:50:122
19410991414,0cyclictest66850irq/125-lan07:47:532
19410991414,0cyclictest66850irq/125-lan07:35:332
19410991414,0cyclictest66850irq/125-lan07:16:132
19410991413,1cyclictest66850irq/125-lan08:00:142
19410991313,0cyclictest66850irq/125-lan07:20:162
19410991212,0cyclictest66850irq/125-lan08:20:152
19410991010,0cyclictest66850irq/125-lan08:15:212
194149998,1cyclictest771-21snmpd09:49:543
194149998,1cyclictest771-21snmpd08:11:003
194149998,1cyclictest771-21snmpd07:54:383
194079998,1cyclictest771-21snmpd12:04:291
194039998,1cyclictest771-21snmpd10:27:590
194039998,1cyclictest771-21snmpd10:27:580
194039998,1cyclictest771-21snmpd09:10:250
194039998,1cyclictest771-21snmpd07:39:170
194039998,1cyclictest769-21lldpd07:15:500
194039998,1cyclictest517-21systemd-resolve10:04:210
194039994,4cyclictest255-20systemd-journal10:45:240
194149988,0cyclictest771-21snmpd11:52:353
194149988,0cyclictest771-21snmpd10:35:363
194149988,0cyclictest771-21snmpd09:20:223
194149988,0cyclictest771-21snmpd07:19:583
194149987,1cyclictest771-21snmpd12:35:303
194149987,1cyclictest771-21snmpd11:30:223
194149987,1cyclictest769-21lldpd09:03:243
194149987,1cyclictest524-21avahi-daemon12:18:053
194149987,1cyclictest29639-21fschecks_time09:40:143
194149986,2cyclictest771-21snmpd12:20:243
194149986,2cyclictest771-21snmpd08:33:393
194149986,2cyclictest771-21snmpd07:12:253
194149985,3cyclictest771-21snmpd11:25:013
194149985,3cyclictest771-21snmpd07:22:013
194079988,0cyclictest769-21lldpd11:18:361
194079988,0cyclictest769-21lldpd10:44:571
194079987,1cyclictest771-21snmpd10:21:431
194079987,1cyclictest771-21snmpd08:26:101
194079987,1cyclictest769-21lldpd10:00:331
194079986,2cyclictest771-21snmpd12:25:221
194079986,2cyclictest771-21snmpd07:43:571
194079986,2cyclictest771-21snmpd07:31:191
194039988,0cyclictest771-21snmpd11:58:300
194039988,0cyclictest771-21snmpd11:31:040
194039988,0cyclictest771-21snmpd10:44:060
194039988,0cyclictest771-21snmpd10:36:530
194039988,0cyclictest771-21snmpd08:38:240
194039988,0cyclictest769-21lldpd09:54:000
194039988,0cyclictest769-21lldpd09:40:230
194039988,0cyclictest769-21lldpd09:21:200
194039988,0cyclictest769-21lldpd08:50:050
194039988,0cyclictest769-21lldpd07:40:500
194039988,0cyclictest769-21lldpd07:28:040
194039987,1cyclictest771-21snmpd08:26:150
194039987,1cyclictest769-21lldpd12:20:590
194039987,1cyclictest769-21lldpd11:25:430
194039987,1cyclictest769-21lldpd09:37:040
194039987,1cyclictest524-21avahi-daemon11:21:590
194039987,1cyclictest517-21systemd-resolve09:26:250
194039986,2cyclictest771-21snmpd11:39:520
194039986,2cyclictest771-21snmpd08:23:180
194149977,0cyclictest771-21snmpd08:17:483
194149977,0cyclictest771-21snmpd07:33:103
194149977,0cyclictest769-21lldpd10:22:193
194149977,0cyclictest769-21lldpd10:17:553
194149977,0cyclictest769-21lldpd09:39:203
194149977,0cyclictest769-21lldpd08:23:453
194149976,1cyclictest524-21avahi-daemon12:07:523
194149976,1cyclictest524-21avahi-daemon10:28:073
194149976,1cyclictest524-21avahi-daemon10:28:063
194149975,2cyclictest771-21snmpd12:28:433
194149975,2cyclictest771-21snmpd08:55:323
194149975,2cyclictest20738-21cat11:55:143
194149974,2cyclictest586-21rs:main3
194079977,0cyclictest771-21snmpd11:27:171
194079977,0cyclictest771-21snmpd11:11:371
194079977,0cyclictest771-21snmpd11:03:271
194079977,0cyclictest771-21snmpd07:21:431
194079977,0cyclictest769-21lldpd12:24:411
194079977,0cyclictest769-21lldpd11:32:541
194079977,0cyclictest769-21lldpd10:30:001
194079977,0cyclictest769-21lldpd10:29:591
194079977,0cyclictest769-21lldpd10:15:211
194079976,1cyclictest771-21snmpd11:53:491
194079976,1cyclictest769-21lldpd09:33:281
194079976,1cyclictest524-21avahi-daemon09:45:271
194079976,1cyclictest517-21systemd-resolve09:44:281
194079974,3cyclictest11769-21ntp_states11:45:201
194079974,2cyclictest255-20systemd-journal10:07:501
194079973,3cyclictest4532-21ssh11:37:171
194079973,3cyclictest11106-21ssh09:18:421
194079972,4cyclictest1475-21cat12:39:591
194039977,0cyclictest771-21snmpd12:01:300
194039977,0cyclictest769-21lldpd11:12:050
194039977,0cyclictest769-21lldpd09:05:340
194039976,1cyclictest771-21snmpd11:03:090
194039976,1cyclictest771-21snmpd10:50:530
194039976,1cyclictest771-21snmpd10:14:320
194039976,1cyclictest771-21snmpd10:06:520
194039975,2cyclictest771-21snmpd12:25:190
194039975,2cyclictest771-21snmpd12:18:270
194039975,2cyclictest771-21snmpd11:09:120
194039975,2cyclictest771-21snmpd08:46:240
194039975,2cyclictest771-21snmpd08:41:270
194039974,2cyclictest255-20systemd-journal09:17:170
194039973,3cyclictest27673-21apt-get07:30:120
194149966,0cyclictest771-21snmpd11:02:093
194149966,0cyclictest769-21lldpd09:08:143
194149966,0cyclictest769-21lldpd08:06:553
194149966,0cyclictest769-21lldpd07:55:353
194149966,0cyclictest524-21avahi-daemon07:29:073
194149965,1cyclictest771-21snmpd10:57:393
194149965,1cyclictest769-21lldpd10:44:393
194149965,1cyclictest584-21in:imuxsock09:16:393
194149965,1cyclictest524-21avahi-daemon11:14:333
194149965,1cyclictest524-21avahi-daemon09:31:393
194149965,1cyclictest517-21systemd-resolve11:42:373
194149964,2cyclictest7877-21cat08:00:143
194149964,2cyclictest771-21snmpd07:46:023
194149964,2cyclictest771-21snmpd07:38:443
194149964,0cyclictest771-21snmpd11:36:323
194149963,3cyclictest550-21NetworkManager10:07:103
194149963,2cyclictest586-21rs:main3
194149963,2cyclictest255-20systemd-journal10:50:513
194149963,2cyclictest255-20systemd-journal09:52:533
194149963,2cyclictest15653-21ssh10:00:423
194149962,3cyclictest26917-21ssh12:33:193
194079966,0cyclictest771-21snmpd12:06:201
194079966,0cyclictest771-21snmpd10:54:181
194079966,0cyclictest771-21snmpd07:11:441
194079966,0cyclictest769-21lldpd10:56:201
194079966,0cyclictest769-21lldpd09:01:341
194079965,1cyclictest771-21snmpd07:25:291
194079965,1cyclictest524-21avahi-daemon08:30:221
194079965,1cyclictest524-21avahi-daemon07:45:121
194079965,1cyclictest4262-21gltestperf09:10:181
194079965,1cyclictest255-20systemd-journal10:49:141
194079965,1cyclictest23761-21gltestperf10:10:141
194079965,1cyclictest0-21swapper/111:05:171
194079964,2cyclictest771-21snmpd09:55:461
194079964,2cyclictest771-21snmpd08:05:021
194079964,2cyclictest771-21snmpd07:37:251
194079964,2cyclictest5917-21cat09:50:141
194079963,3cyclictest21901-21fw_forwarded_lo07:15:141
194079963,2cyclictest586-21rs:main1
194079963,2cyclictest586-21rs:main1
194079963,2cyclictest255-20systemd-journal09:37:211
194079963,2cyclictest21790-21ssh11:55:311
194079962,3cyclictest6879-21ssh11:40:161
194079962,3cyclictest16812-21ssh10:38:321
194039966,0cyclictest771-21snmpd10:16:350
194039966,0cyclictest769-21lldpd08:10:500
194039965,1cyclictest771-21snmpd08:18:030
194039965,1cyclictest524-21avahi-daemon09:56:540
194039965,1cyclictest517-21systemd-resolve08:59:470
194039964,2cyclictest771-21snmpd10:22:220
194039964,2cyclictest771-21snmpd08:03:410
194039964,2cyclictest771-21snmpd07:51:560
194039964,2cyclictest771-21snmpd07:20:250
194039964,2cyclictest550-21NetworkManager12:14:200
194039963,2cyclictest29578-21ntpq12:35:190
194149955,0cyclictest769-21lldpd11:46:123
194149955,0cyclictest769-21lldpd11:21:413
194149953,2cyclictest771-21snmpd10:45:473
194149952,3cyclictest550-21NetworkManager12:00:293
194149952,2cyclictest9940-21ssh11:06:333
194149952,2cyclictest9728-21apt-get09:55:143
194149952,2cyclictest749-21ntpd11:15:163
194149952,2cyclictest586-21rs:main3
194149952,2cyclictest4971-21ssh09:10:243
194149952,2cyclictest28304-21cut08:50:153
194149952,2cyclictest27355-21ssh10:14:523
194149951,3cyclictest25916-21perl08:45:143
194079955,0cyclictest771-21snmpd08:14:571
194079953,2cyclictest771-21snmpd08:35:031
194079953,2cyclictest550-21NetworkManager08:45:181
194079953,2cyclictest255-20systemd-journal08:52:371
194079952,2cyclictest8093-21ssh12:15:481
194079952,2cyclictest749-21ntpd08:40:181
194079952,2cyclictest24246-21ssh11:23:361
194079952,2cyclictest2379-21grep09:05:151
194079952,2cyclictest10150-21gcc10:30:181
194079951,3cyclictest12383-21dpkg09:20:001
194079950,4cyclictest17824-21ntpq09:25:191
194039954,1cyclictest16369-21ntpq11:50:190
194039953,2cyclictest586-21rs:main0
194039953,2cyclictest25407-21ssh12:31:240
194039953,2cyclictest0-21swapper/010:30:070
194039952,2cyclictest2840-21sh09:45:410
194039952,2cyclictest22465-21sed09:30:220
194039952,2cyclictest20784-21unixbench-2d08:30:210
194039952,2cyclictest17741-21awk11:15:200
194039952,2cyclictest12707-21rm11:45:570
194039951,3cyclictest32513-21sh10:55:220
194039951,3cyclictest31794-21ssh12:07:320
194039950,4cyclictest446-21ntpq09:00:170
194149942,2cyclictest1749-21tail12:10:143
194149940,4cyclictest32576-21ntpq07:40:163
194149940,3cyclictest255-20systemd-journal08:25:153
194079944,0cyclictest3477-21cpuspeed_turbos07:50:131
194079942,2cyclictest550-21NetworkManager08:15:141
194079942,2cyclictest30641-21gzip08:55:191
194079941,2cyclictest6465-21grep07:55:191
194079941,2cyclictest16271-21ld08:20:191
194079940,3cyclictest7617-21diskstats08:00:131
194039942,2cyclictest9435-21sshd08:05:120
194039942,0cyclictest0-21swapper/007:59:310
194039940,4cyclictest20441-21ntpq07:10:180
194039930,2cyclictest1440-21awk07:45:120
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional