You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-29 - 01:25

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot5s.osadl.org (updated Fri Aug 28, 2026 12:44:21)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1235099250,24cyclictest0-21swapper/309:52:213
1235099250,1cyclictest21263-21munin-run08:35:013
12341992524,1cyclictest0-21swapper/010:00:050
1234199250,24cyclictest3814-21ssh10:33:060
1234199250,24cyclictest0-21swapper/009:14:200
1234199250,23cyclictest313-21systemd-journal09:20:010
1234199250,1cyclictest0-21swapper/011:15:000
1234199250,0cyclictest0-21swapper/010:18:120
1235099240,23cyclictest6069-21diskmemload10:37:243
1235099240,23cyclictest6069-21diskmemload10:18:183
1235099240,23cyclictest2584-21perf11:15:003
1234999241,22cyclictest0-21swapper/210:20:002
1234999240,23cyclictest6069-21diskmemload10:32:142
1234999240,23cyclictest0-21swapper/209:50:242
1234999240,0cyclictest6069-21diskmemload09:12:222
1234999240,0cyclictest0-21swapper/209:20:062
12345992424,0cyclictest6069-21diskmemload10:36:581
12345992424,0cyclictest0-21swapper/110:49:031
1234599240,24cyclictest6069-21diskmemload09:45:341
1234599240,24cyclictest0-21swapper/109:43:401
12341992424,0cyclictest6069-21diskmemload10:20:330
12341992424,0cyclictest141rcuc/009:52:010
12341992424,0cyclictest141rcuc/009:34:580
1234199240,3cyclictest313-21systemd-journal12:20:000
12350992323,0cyclictest30347-21rm10:27:363
12350992323,0cyclictest0-21swapper/310:50:573
12350992323,0cyclictest0-21swapper/309:55:333
12350992323,0cyclictest0-21swapper/309:00:513
12350992321,1cyclictest29055-21perf12:10:003
1235099230,2cyclictest6069-21diskmemload10:00:543
1235099230,23cyclictest0-21swapper/310:13:033
1235099230,22cyclictest8126-21perf09:40:003
1235099230,22cyclictest26841-21sh07:40:013
1235099230,22cyclictest16695-21latency_hist10:45:013
1235099230,0cyclictest6069-21diskmemload09:17:003
12349992323,0cyclictest0-21swapper/210:25:242
12349992322,1cyclictest6069-21diskmemload09:19:352
12349992320,2cyclictest25896-21rm09:55:092
12349992320,2cyclictest24489-21ssh10:22:082
1234999230,2cyclictest1835-21ssh10:01:302
1234999230,23cyclictest0-21swapper/211:10:522
1234999230,23cyclictest0-21swapper/209:31:412
1234999230,22cyclictest8543-21ssh10:08:082
1234999230,22cyclictest7305-21ssh09:39:022
1234999230,22cyclictest606-21dbus-daemon08:35:012
1234999230,22cyclictest17077-21rm10:45:132
1234999230,22cyclictest13483-21sh10:12:162
1234999230,0cyclictest0-21swapper/210:42:402
12345992323,0cyclictest6069-21diskmemload09:19:051
12345992323,0cyclictest0-21swapper/110:25:091
12345992321,1cyclictest455-21apt-get11:10:531
12345992321,1cyclictest10728-21apt-get10:40:021
1234599230,2cyclictest23474-21sh09:52:081
1234599230,23cyclictest0-21swapper/110:15:331
1234599230,23cyclictest0-21swapper/110:06:421
1234599230,23cyclictest0-21swapper/110:02:471
1234599230,23cyclictest0-21swapper/109:30:061
1234599230,22cyclictest11066-21basename12:40:011
1234599230,0cyclictest0-21swapper/110:12:541
1234599230,0cyclictest0-21swapper/109:28:421
1234599230,0cyclictest0-21swapper/109:20:281
1234599230,0cyclictest0-21swapper/109:10:201
12341992323,0cyclictest6069-21diskmemload09:48:570
12341992323,0cyclictest0-21swapper/010:36:420
12341992323,0cyclictest0-21swapper/009:35:560
12341992323,0cyclictest0-21swapper/008:00:510
12341992320,2cyclictest10444-21ssh10:10:210
1234199230,22cyclictest5152-21sh10:05:490
1234199230,22cyclictest23364-21dump-pmu-power09:25:010
1234199230,22cyclictest21866-21cron07:30:000
1234199230,22cyclictest0-21swapper/010:46:180
1234199230,1cyclictest8844-21perf08:10:000
12350992222,0cyclictest0-21swapper/310:21:053
12350992221,1cyclictest6069-21diskmemload10:31:543
12350992220,1cyclictest0-21swapper/309:25:293
12350992220,1cyclictest0-21swapper/309:20:083
12350992220,1cyclictest0-21swapper/309:15:013
1235099220,22cyclictest13827-21ssh09:44:463
1235099220,21cyclictest606-21dbus-daemon12:20:003
1235099220,21cyclictest4837-21ssh10:05:193
1235099220,21cyclictest32590-21perf11:10:003
1235099220,21cyclictest30214-21awk11:05:013
1235099220,21cyclictest17018-21munin-run11:45:013
1235099220,21cyclictest1648-21idleruntime-cro07:55:003
1235099220,21cyclictest14603-21turbostat11:40:013
1235099220,21cyclictest14454-21ssh09:45:233
1235099220,21cyclictest0-21swapper/307:25:003
1235099220,1cyclictest3854-21idleruntime-cro12:25:013
1235099220,1cyclictest0-21swapper/309:34:573
12349992222,0cyclictest1782-21apt-get12:20:532
12349992222,0cyclictest0-21swapper/209:28:112
12349992221,1cyclictest0-21swapper/210:39:022
12349992221,0cyclictest29576-21apt-get07:45:502
12349992220,1cyclictest18837-21sh08:30:012
1234999220,21cyclictest0-21swapper/211:10:002
1234999220,1cyclictest14677-21apt07:15:002
1234999220,1cyclictest0-21swapper/209:44:142
1234999220,0cyclictest0-21swapper/210:50:282
1234999220,0cyclictest0-21swapper/209:46:542
12345992221,1cyclictest6069-21diskmemload10:30:491
12345992220,1cyclictest17052-21cron07:20:001
12345992220,1cyclictest0-21swapper/109:55:521
1234599221,0cyclictest12512-21apt-get11:35:521
1234599220,21cyclictest0-21swapper/112:30:521
1234599220,21cyclictest0-21swapper/110:50:591
1234599220,21cyclictest0-21swapper/107:20:511
1234599220,1cyclictest21379-21ssh10:20:141
1234599220,1cyclictest21235-21apt-get08:35:021
1234599220,1cyclictest0-21swapper/112:15:511
12341992222,0cyclictest6069-21diskmemload10:50:270
12341992221,1cyclictest0-21swapper/010:25:040
12341992220,1cyclictest31454-21apt12:15:000
12341992220,1cyclictest31294-21perf08:55:000
12341992220,1cyclictest13618-21cron08:20:030
12341992220,1cyclictest0-21swapper/009:43:020
12341992219,2cyclictest21248-21awk08:35:010
1234199220,21cyclictest6069-21diskmemload09:55:160
1234199220,21cyclictest606-21dbus-daemon09:00:000
1234199220,21cyclictest0-21swapper/012:20:530
1234199220,21cyclictest0-21swapper/011:20:530
1234199220,21cyclictest0-21swapper/009:25:550
1234199220,1cyclictest0-21swapper/010:42:390
12350992121,0cyclictest19717-21apt-get11:50:503
12350992119,1cyclictest0-21swapper/307:50:013
1235099210,21cyclictest14694-21munin-run07:15:003
1235099210,21cyclictest0-21swapper/311:55:553
1235099210,21cyclictest0-21swapper/311:30:513
1235099210,0cyclictest0-21swapper/308:20:033
1235099210,0cyclictest0-21swapper/308:05:513
1235099210,0cyclictest0-21swapper/308:00:513
1235099210,0cyclictest0-21swapper/307:25:513
12349992120,1cyclictest24713-21apt-get07:35:512
12349992119,1cyclictest9767-21idleruntime-cro11:30:002
12349992119,1cyclictest21867-21apt-get08:35:022
12349992119,1cyclictest19816-21apt-get07:25:512
12349992119,1cyclictest1-21systemd11:40:012
1234999210,20cyclictest31312-21munin-run08:55:002
12345992121,0cyclictest26984-21apt-get12:05:521
12345992119,1cyclictest0-21swapper/111:35:001
1234599210,20cyclictest0-21swapper/111:25:511
1234599210,1cyclictest21903-21munin-run07:30:001
1234599210,0cyclictest0-21swapper/108:05:511
12341992120,1cyclictest0-21swapper/008:15:510
12341992120,0cyclictest0-21swapper/011:55:550
1234199210,21cyclictest0-21swapper/008:42:290
1234199210,21cyclictest0-21swapper/007:10:500
1234199210,20cyclictest0-21swapper/012:05:520
1234199210,1cyclictest27778-21turbostat.cron11:00:010
1234199210,0cyclictest0-21swapper/011:25:510
12350992020,0cyclictest0-21swapper/312:35:013
12350992020,0cyclictest0-21swapper/311:45:513
12350992020,0cyclictest0-21swapper/308:50:523
12350992020,0cyclictest0-21swapper/308:35:513
12350992020,0cyclictest0-21swapper/307:55:513
12350992019,0cyclictest22214-21apt-get07:30:513
12350992019,0cyclictest0-21swapper/310:55:523
1235099200,0cyclictest0-21swapper/308:25:513
12349992020,0cyclictest27184-21apt-get07:40:512
12349992020,0cyclictest0-21swapper/208:40:512
12349992019,1cyclictest6768-21apt-get08:05:512
12349992018,1cyclictest606-21dbus-daemon12:00:002
1234999200,2cyclictest13631-21sh08:20:002
1234999200,0cyclictest0-21swapper/212:25:522
12345992020,0cyclictest1956-21apt-get07:55:511
12345992020,0cyclictest0-21swapper/111:55:551
12345992020,0cyclictest0-21swapper/108:55:531
12345992020,0cyclictest0-21swapper/107:40:511
12345992019,0cyclictest28127-21apt-get11:00:511
12345992018,1cyclictest26515-21awk08:45:001
12345992017,2cyclictest313-21systemd-journal08:50:001
12341992020,0cyclictest0-21swapper/011:15:520
12341992020,0cyclictest0-21swapper/007:20:510
12341992019,0cyclictest0-21swapper/012:30:520
12341992018,1cyclictest11252-21grep08:15:000
1234199200,20cyclictest0-21swapper/008:35:020
12350991919,0cyclictest0-21swapper/312:25:523
12350991919,0cyclictest0-21swapper/311:20:533
12350991919,0cyclictest0-21swapper/307:15:503
12349991919,0cyclictest24537-21apt-get12:00:532
12349991919,0cyclictest17325-21apt-get11:45:512
12349991919,0cyclictest15001-21apt-get07:15:502
12349991919,0cyclictest0-21swapper/211:40:532
12345991919,0cyclictest0-21swapper/111:20:531
12345991918,1cyclictest13669-21perf08:20:001
12345991917,1cyclictest21812-21munin-run11:55:011
12345991917,1cyclictest11212-21cron08:15:001
12341991919,0cyclictest0-21swapper/011:40:530
12341991919,0cyclictest0-21swapper/009:05:520
12341991917,1cyclictest0-21swapper/011:05:010
12341991916,2cyclictest0-21swapper/012:00:530
12350991818,0cyclictest0-21swapper/312:35:513
12350991818,0cyclictest0-21swapper/311:15:523
12349991818,0cyclictest0-21swapper/208:45:522
12349991818,0cyclictest0-21swapper/207:50:512
12349991817,1cyclictest11237-21apt08:15:002
12349991817,0cyclictest5320-21apt-get11:20:532
12345991818,0cyclictest4175-21apt-get12:25:531
12345991818,0cyclictest0-21swapper/112:20:541
12345991818,0cyclictest0-21swapper/108:30:531
12345991817,1cyclictest0-21swapper/111:45:511
12345991816,1cyclictest22-21ksoftirqd/109:10:011
12341991818,0cyclictest0-21swapper/007:15:500
12341991816,1cyclictest0-21swapper/007:45:500
12341991815,2cyclictest0-21swapper/007:36:000
12341991814,2cyclictest0-21swapper/009:00:510
12350991717,0cyclictest0-21swapper/312:00:523
12349991717,0cyclictest0-21swapper/209:00:512
12349991717,0cyclictest0-21swapper/207:30:512
12349991716,1cyclictest6443-21turbostat08:05:002
12349991716,1cyclictest1432-21perf12:20:002
12349991715,1cyclictest10106-21apt-get11:30:512
1234999170,0cyclictest0-21swapper/212:30:522
12345991717,0cyclictest14378-21apt-get08:20:511
12345991717,0cyclictest0-21swapper/108:00:511
12345991716,1cyclictest25713-21apt-get10:55:531
12341991717,0cyclictest0-21swapper/011:30:510
12341991717,0cyclictest0-21swapper/011:05:510
12341991715,1cyclictest8967-21apt-get12:35:520
1235099160,15cyclictest6103-21munin-run09:10:013
12349991616,0cyclictest0-21swapper/212:05:522
12349991616,0cyclictest0-21swapper/211:50:502
12349991616,0cyclictest0-21swapper/208:55:532
12345991616,0cyclictest0-21swapper/111:15:521
12345991616,0cyclictest0-21swapper/107:45:511
12345991616,0cyclictest0-21swapper/107:35:511
12345991616,0cyclictest0-21swapper/107:10:501
12345991614,1cyclictest24305-21munin-run07:35:011
12345991614,1cyclictest2263-21hddtemp_smartct09:00:531
12345991614,1cyclictest18866-21awk08:30:011
12345991614,1cyclictest14933-21apt-get11:40:531
12341991616,0cyclictest0-21swapper/008:45:510
12341991614,1cyclictest21798-21perf11:55:010
12341991614,1cyclictest1683-21latency_hist07:55:000
12341991613,2cyclictest22438-21cstates07:30:520
12350991514,1cyclictest0-21swapper/308:45:393
1235099150,1cyclictest13590-21taskset08:19:453
1235099150,11cyclictest0-21swapper/307:40:513
12349991515,0cyclictest2899-21apt-get11:15:522
12341991515,0cyclictest0-21swapper/012:25:520
12341991515,0cyclictest0-21swapper/011:35:520
1234199150,14cyclictest18856-21munin-plugin-st08:30:010
12350991414,0cyclictest0-21swapper/308:55:123
12350991414,0cyclictest0-21swapper/308:40:063
12350991414,0cyclictest0-21swapper/308:10:143
1235099140,14cyclictest0-21swapper/312:11:053
1235099140,0cyclictest0-21swapper/311:25:313
12349991414,0cyclictest0-21swapper/207:55:112
12349991414,0cyclictest0-21swapper/207:20:472
1234999140,14cyclictest29122-21strings11:00:552
1234999140,14cyclictest0-21swapper/212:35:102
1234999140,14cyclictest0-21swapper/208:20:482
1234999140,0cyclictest0-21swapper/212:10:342
1234999140,0cyclictest0-21swapper/209:05:242
12345991414,0cyclictest0-21swapper/112:10:071
1234599140,0cyclictest0-21swapper/108:50:051
1234199140,14cyclictest0-21swapper/007:55:200
1234199140,14cyclictest0-21swapper/007:40:150
1234199140,0cyclictest0-21swapper/011:45:300
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional