You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-29 - 17:40

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot5s.osadl.org (updated Mon Jun 29, 2026 12:44:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
28954993021,4cyclictest23070-21diskmemload11:15:312
2895499280,24cyclictest0-21swapper/209:18:332
28954992622,3cyclictest7898-21apt-get08:40:022
28954992621,4cyclictest7724-21ssh10:49:542
2895499260,25cyclictest0-21swapper/211:13:052
2895499260,1cyclictest0-21swapper/211:28:262
2895499260,1cyclictest0-21swapper/210:25:082
2895599250,24cyclictest25544-21ssh12:01:083
28954992522,3cyclictest0-21swapper/212:01:432
28954992522,3cyclictest0-21swapper/210:55:512
28954992520,4cyclictest17606-21ssh09:33:002
28954992520,4cyclictest0-21swapper/209:49:402
28954992518,6cyclictest31931-21grep12:35:592
2895499250,24cyclictest0-21swapper/211:46:222
2895499250,24cyclictest0-21swapper/209:20:592
2895399251,23cyclictest20847-21mii-tool11:00:561
2895399250,24cyclictest23070-21diskmemload10:55:101
2895399250,24cyclictest13097-21apt-get08:50:021
2895399250,24cyclictest0-21swapper/111:45:561
28952992523,1cyclictest24831-21munin-plugin-st12:30:000
28955992423,1cyclictest23070-21diskmemload12:31:303
2895599240,24cyclictest0-21swapper/309:20:593
2895599240,23cyclictest0-21swapper/310:39:263
2895599240,23cyclictest0-21swapper/307:30:013
2895599240,0cyclictest23070-21diskmemload11:47:553
28954992424,0cyclictest0-21swapper/211:37:462
28954992421,2cyclictest16491-21taskset12:21:142
28954992421,2cyclictest0-21swapper/211:51:032
28954992420,3cyclictest0-21swapper/212:31:152
28954992420,3cyclictest0-21swapper/210:06:282
28954992419,4cyclictest6486-21ssh10:20:392
28954992418,5cyclictest0-21swapper/212:09:092
2895499240,24cyclictest0-21swapper/211:42:142
2895499240,23cyclictest23070-21diskmemload09:55:382
2895499240,23cyclictest0-21swapper/212:10:042
2895499240,23cyclictest0-21swapper/209:41:112
2895499240,21cyclictest0-21swapper/210:12:552
2895499240,1cyclictest23070-21diskmemload09:52:372
2895499240,0cyclictest0-21swapper/210:40:462
2895499240,0cyclictest0-21swapper/209:27:542
28953992422,1cyclictest0-21swapper/109:45:281
2895399240,23cyclictest28482-21apt-get10:40:021
2895399240,23cyclictest17292-21sh12:22:261
2895399240,23cyclictest12359-21apt-get10:25:561
2895399240,23cyclictest0-21swapper/111:16:461
2895399240,23cyclictest0-21swapper/110:17:561
28952992422,1cyclictest62550irq/127-eno112:24:320
2895299242,1cyclictest1334-21latency_hist07:20:010
28952992421,2cyclictest31990-21ssh09:45:540
28952992421,2cyclictest0-21swapper/009:28:560
2895299240,24cyclictest0-21swapper/010:20:120
2895299240,23cyclictest23070-21diskmemload11:43:470
2895299240,23cyclictest23070-21diskmemload10:56:540
2895299240,22cyclictest7680-21diskmemload10:49:510
2895299240,21cyclictest3733-21awk07:25:010
28955992323,0cyclictest23070-21diskmemload11:53:453
28955992323,0cyclictest23070-21diskmemload10:57:433
28955992323,0cyclictest0-21swapper/311:55:353
28955992323,0cyclictest0-21swapper/309:50:513
2895599232,21cyclictest23070-21diskmemload12:28:523
28955992320,2cyclictest11023-21ssh10:51:423
28955992320,2cyclictest10705-21perf08:45:003
2895599230,2cyclictest3438-21ssh12:39:413
2895599230,2cyclictest15811-21ssh11:25:443
2895599230,23cyclictest0-21swapper/312:05:203
2895599230,22cyclictest4073-21diskmemload10:17:443
2895599230,22cyclictest32204-21ssh10:14:313
2895599230,22cyclictest313-21systemd-journal10:30:013
2895599230,22cyclictest23070-21diskmemload09:10:333
2895599230,22cyclictest22959-21awk11:31:033
2895599230,22cyclictest21245-21ssh10:32:563
2895599230,22cyclictest13740-21sh09:30:033
2895599230,0cyclictest0-21swapper/311:14:413
2895599230,0cyclictest0-21swapper/310:24:273
2895599230,0cyclictest0-21swapper/309:27:393
28954992323,0cyclictest23070-21diskmemload11:57:292
28954992323,0cyclictest23070-21diskmemload10:02:382
28954992323,0cyclictest0-21swapper/211:20:142
28954992321,2cyclictest0-21swapper/209:35:332
28954992321,1cyclictest23565-21ssh09:10:192
28954992320,3cyclictest0-21swapper/210:53:132
2895499230,23cyclictest0-21swapper/211:30:072
2895499230,22cyclictest23465-21ssh11:03:072
2895499230,22cyclictest23070-21diskmemload10:15:562
2895499230,22cyclictest20113-21/usr/sbin/munin12:25:592
2895499230,0cyclictest23070-21diskmemload10:35:412
2895499230,0cyclictest0-21swapper/211:05:022
28953992323,0cyclictest23070-21diskmemload11:54:331
28953992323,0cyclictest23070-21diskmemload11:20:041
28953992323,0cyclictest23070-21diskmemload10:31:231
28953992323,0cyclictest23070-21diskmemload09:39:431
28953992323,0cyclictest0-21swapper/110:46:581
28953992323,0cyclictest0-21swapper/110:35:201
28953992323,0cyclictest0-21swapper/109:26:331
28953992322,1cyclictest0-21swapper/111:12:261
28953992322,1cyclictest0-21swapper/109:55:321
2895399230,23cyclictest0-21swapper/112:17:051
2895399230,23cyclictest0-21swapper/109:53:151
2895399230,22cyclictest30963-21taskset11:38:251
2895399230,22cyclictest30842-21diskmemload12:35:071
2895399230,22cyclictest2753-21sh11:41:261
2895399230,22cyclictest23580-21sh09:10:201
2895399230,22cyclictest23070-21diskmemload11:31:161
2895399230,22cyclictest21055-21latency12:25:591
2895399230,22cyclictest18717-21ssh11:27:171
2895399230,22cyclictest17354-21sh09:32:411
2895399230,22cyclictest0-21swapper/111:08:111
2895399230,22cyclictest0-21swapper/110:50:261
2895399230,22cyclictest0-21swapper/110:13:071
2895399230,1cyclictest25906-21grep08:10:011
2895399230,1cyclictest0-21swapper/109:23:211
2895399230,0cyclictest23070-21diskmemload12:04:291
2895399230,0cyclictest23070-21diskmemload10:05:391
28952992323,0cyclictest23070-21diskmemload09:56:230
28952992323,0cyclictest0-21swapper/012:13:010
28952992323,0cyclictest0-21swapper/011:20:060
28952992323,0cyclictest0-21swapper/011:15:250
28952992323,0cyclictest0-21swapper/011:08:150
28952992323,0cyclictest0-21swapper/011:01:290
28952992323,0cyclictest0-21swapper/010:25:290
28952992323,0cyclictest0-21swapper/010:09:360
28952992323,0cyclictest0-21swapper/009:36:030
28952992321,1cyclictest0-21swapper/011:10:590
28952992320,2cyclictest7275-21sh09:24:260
28952992320,2cyclictest28300-21munin-run08:15:010
2895299230,2cyclictest2079-21ssh10:44:590
2895299230,22cyclictest7543-21ssh09:51:470
2895299230,22cyclictest28198-21sh10:39:430
2895299230,22cyclictest23070-21diskmemload11:55:460
2895299230,22cyclictest20767-21ssh10:32:210
2895299230,22cyclictest17039-21ssh11:25:590
2895299230,22cyclictest11036-21perf07:40:000
2895299230,22cyclictest0-21swapper/011:54:440
2895299230,22cyclictest0-21swapper/011:50:010
2895299230,0cyclictest23070-21diskmemload12:37:590
2895299230,0cyclictest0-21swapper/010:15:580
2895299230,0cyclictest0-21swapper/009:33:110
28955992222,0cyclictest0-21swapper/310:07:073
28955992222,0cyclictest0-21swapper/309:40:113
2895599222,19cyclictest606-21dbus-daemon08:20:013
28955992221,1cyclictest1458-21ssh09:46:183
28955992221,1cyclictest0-21swapper/311:05:333
2895599220,22cyclictest23070-21diskmemload11:37:083
2895599220,22cyclictest0-21swapper/311:40:113
2895599220,22cyclictest0-21swapper/311:04:093
2895599220,21cyclictest8708-21ssh12:15:483
2895599220,21cyclictest8539-21munin-run07:35:003
2895599220,21cyclictest23070-21diskmemload09:15:313
2895599220,21cyclictest16211-21apt-get08:55:563
2895599220,21cyclictest0-21swapper/311:16:443
2895599220,21cyclictest0-21swapper/309:55:043
2895599220,1cyclictest17152-21ssh12:22:163
2895599220,1cyclictest10001-21ssh11:20:213
2895599220,0cyclictest0-21swapper/312:10:553
28954992220,1cyclictest7581-21apt-get12:15:022
28954992220,1cyclictest313-21systemd-journal08:40:002
28954992220,1cyclictest18440-21sh10:30:572
28954992220,1cyclictest1340-21munin-run07:20:012
28954992219,2cyclictest606-21dbus-daemon08:00:002
28954992218,3cyclictest29267-21apt-get07:10:552
28954992218,3cyclictest0-21swapper/207:35:562
28954992217,4cyclictest0-21swapper/208:25:002
2895499220,1cyclictest0-21swapper/207:50:002
28953992222,0cyclictest23070-21diskmemload11:56:001
28953992221,0cyclictest0-21swapper/109:40:021
28953992220,1cyclictest0-21swapper/112:34:431
28953992220,1cyclictest0-21swapper/112:05:111
2895399221,20cyclictest8622-21apt-get08:40:521
2895399220,22cyclictest0-21swapper/110:00:521
2895399220,21cyclictest9418-21ssh10:22:131
2895399220,21cyclictest18622-21apt-get09:00:531
2895399220,21cyclictest0-21swapper/108:00:541
2895399220,21cyclictest0-21swapper/107:45:531
2895399220,1cyclictest619-21polkitd08:40:001
2895399220,1cyclictest0-21swapper/109:15:371
2895399220,0cyclictest0-21swapper/112:10:431
28952992222,0cyclictest0-21swapper/010:13:330
28952992222,0cyclictest0-21swapper/009:41:430
28952992220,2cyclictest0-21swapper/009:10:060
28952992220,1cyclictest18298-21idleruntime-cro09:00:010
28952992220,1cyclictest11152-21sh10:51:510
28952992219,2cyclictest28860-21cat12:05:010
28952992219,2cyclictest21277-21sh11:30:260
2895299220,21cyclictest7898-21apt-get08:40:020
2895299220,21cyclictest28945-21ssh12:05:070
2895299220,21cyclictest0-21swapper/012:15:560
2895299220,21cyclictest0-21swapper/008:20:550
2895299220,1cyclictest21047-21ssh10:04:430
2895299220,1cyclictest13461-21perf07:45:000
2895299220,1cyclictest0-21swapper/012:31:120
28955992121,0cyclictest18959-21apt-get07:55:553
28955992121,0cyclictest0-21swapper/310:40:453
2895599210,21cyclictest0-21swapper/310:45:563
2895599210,21cyclictest0-21swapper/309:35:193
2895599210,21cyclictest0-21swapper/308:35:563
2895599210,20cyclictest0-21swapper/308:10:563
28954992120,1cyclictest1008-21apt-get08:25:542
28954992119,1cyclictest13712-21apt-get08:50:022
2895499210,0cyclictest0-21swapper/208:45:562
2895499210,0cyclictest0-21swapper/208:45:562
28953992121,0cyclictest0-21swapper/108:20:541
28953992120,0cyclictest1666-21apt-get07:20:561
28953992119,1cyclictest18631-21idleruntime-cro07:55:001
2895399210,20cyclictest16211-21apt-get08:55:561
28952992120,1cyclictest0-21swapper/008:30:530
2895299210,21cyclictest0-21swapper/007:55:550
28955992020,0cyclictest16562-21apt-get07:50:543
28955992020,0cyclictest0-21swapper/307:20:563
2895599200,20cyclictest0-21swapper/309:05:563
2895599200,0cyclictest14167-21apt-get07:45:533
28954992017,2cyclictest28295-21perf08:15:012
28953992020,0cyclictest0-21swapper/109:05:561
28953992020,0cyclictest0-21swapper/107:40:551
28952992020,0cyclictest0-21swapper/008:25:540
28952992020,0cyclictest0-21swapper/007:50:540
28952992018,1cyclictest30720-21sendmail-msp08:20:010
28955991919,0cyclictest0-21swapper/309:00:533
28955991919,0cyclictest0-21swapper/308:50:023
28955991919,0cyclictest0-21swapper/307:40:563
28954991919,0cyclictest0-21swapper/208:15:552
28954991917,1cyclictest0-21swapper/209:00:012
28954991914,1cyclictest3414-21apt-get08:30:532
28954991912,1cyclictest0-21swapper/208:05:012
28953991919,0cyclictest0-21swapper/108:45:561
28953991919,0cyclictest0-21swapper/108:45:561
28953991919,0cyclictest0-21swapper/107:55:551
28953991917,1cyclictest0-21swapper/108:30:001
28952991919,0cyclictest0-21swapper/007:10:550
28954991816,1cyclictest8540-21perf07:35:002
28954991815,2cyclictest13462-21munin-plugin-st07:45:002
28953991818,0cyclictest0-21swapper/107:30:561
28953991818,0cyclictest0-21swapper/107:15:561
28952991818,0cyclictest0-21swapper/007:45:530
28952991816,1cyclictest7857-21cron08:40:000
2895299180,2cyclictest0-21swapper/009:06:060
28955991717,0cyclictest21374-21apt-get08:00:543
2895499170,16cyclictest4450-21ls07:25:532
28953991717,0cyclictest0-21swapper/108:30:531
28953991717,0cyclictest0-21swapper/108:15:541
28953991714,2cyclictest21850irq/141-nvme0q208:15:011
28952991717,0cyclictest0-21swapper/008:00:540
28952991717,0cyclictest0-21swapper/007:30:560
28955991616,0cyclictest0-21swapper/308:45:563
28955991616,0cyclictest0-21swapper/308:45:563
28955991614,1cyclictest31229-21expr08:20:553
28954991614,1cyclictest0-21swapper/207:50:562
2895499160,1cyclictest1886-21cstates07:20:572
2895499160,15cyclictest24247-21diskstats08:05:562
28953991614,1cyclictest0-21swapper/107:15:011
28952991616,0cyclictest0-21swapper/008:45:550
28952991616,0cyclictest0-21swapper/008:45:550
28952991616,0cyclictest0-21swapper/008:05:550
28955991515,0cyclictest0-21swapper/308:05:553
28954991514,1cyclictest0-21swapper/209:05:012
2895499150,1cyclictest0-21swapper/209:10:002
28953991515,0cyclictest0-21swapper/107:35:561
28952991515,0cyclictest0-21swapper/007:25:520
28955991412,1cyclictest4029-21sed08:30:563
2895599140,14cyclictest0-21swapper/308:25:593
2895599140,14cyclictest0-21swapper/307:35:233
2895599140,14cyclictest0-21swapper/307:16:033
2895599140,14cyclictest0-21swapper/307:10:143
28952991414,0cyclictest0-21swapper/009:00:550
2895299140,0cyclictest0-21swapper/008:51:120
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional