You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-19 - 06:03

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Fri Jun 19, 2026 00:44:45)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
203528321000,3sleep20-21swapper/223:15:182
139099900,75rtkit-daemon0-21swapper/319:09:243
194623827439,28sleep00-21swapper/019:05:140
194741526444,14sleep10-21swapper/119:09:121
194731226345,13sleep20-21swapper/219:07:542
19564542490,5sleep33056-21PK-Backend19:30:243
20636342480,3sleep03056-21PK-Backend00:35:190
20433552240,4sleep30-21swapper/323:40:133
194761499244,18cyclictest2062276-21cron00:35:011
1947612992415,7cyclictest436-20systemd-journal00:05:000
19761422230,3sleep33056-21PK-Backend20:25:223
194762299230,20cyclictest3056-21PK-Backend21:10:103
1947614992315,6cyclictest2016674-21cron22:24:591
1947614992315,6cyclictest2004543-21cron21:50:001
194761499230,21cyclictest1967858-21chrt20:04:411
194761299231,20cyclictest2046425-21awk23:50:000
139099230,15rtkit-daemon3056-21PK-Backend00:19:363
1947622992220,1cyclictest3056-21PK-Backend23:11:583
1947618992220,1cyclictest0-21swapper/222:29:272
1947618992219,2cyclictest0-21swapper/222:14:122
1947612992220,1cyclictest0-21swapper/023:57:360
1947612992220,1cyclictest0-21swapper/023:43:220
1947612992220,1cyclictest0-21swapper/021:22:090
1947612992220,1cyclictest0-21swapper/021:07:550
1947612992214,6cyclictest1962329-21http19:48:510
194761299221,19cyclictest1964168-21perf19:54:590
194761299220,7cyclictest2039490-21turbostat23:30:010
194761299220,21cyclictest0-21swapper/019:23:390
139099220,15rtkit-daemon0-21swapper/220:36:312
20225222210,8chrt2022130-21/usr/sbin/munin22:40:131
1947622992119,1cyclictest3056-21PK-Backend22:07:093
1947622992119,1cyclictest3056-21PK-Backend00:20:403
1947622992119,1cyclictest0-21swapper/322:28:393
1947622992118,2cyclictest3056-21PK-Backend20:24:423
1947622992118,2cyclictest3056-21PK-Backend20:12:193
1947622992113,6cyclictest1949719-21ntp_states19:15:023
1947622992111,7cyclictest1990689-21cron21:09:593
194762299210,20cyclictest3056-21PK-Backend20:57:143
194762299210,20cyclictest0-21swapper/321:21:343
194762299210,20cyclictest0-21swapper/319:35:073
194762299210,19cyclictest3056-21PK-Backend21:38:503
194762299210,19cyclictest3056-21PK-Backend00:13:563
1947618992119,1cyclictest0-21swapper/223:53:212
1947618992119,1cyclictest0-21swapper/223:25:582
1947618992119,1cyclictest0-21swapper/223:13:202
1947618992119,1cyclictest0-21swapper/222:09:332
1947618992119,1cyclictest0-21swapper/221:51:552
1947618992119,1cyclictest0-21swapper/221:32:352
1947618992119,1cyclictest0-21swapper/221:25:392
1947618992119,1cyclictest0-21swapper/220:58:352
1947618992119,1cyclictest0-21swapper/220:07:522
1947618992119,1cyclictest0-21swapper/219:41:502
1947618992119,1cyclictest0-21swapper/200:23:032
1947618992119,1cyclictest0-21swapper/200:03:432
1947618992118,2cyclictest3056-21PK-Backend21:46:062
194761899210,20cyclictest0-21swapper/220:01:302
194761899210,19cyclictest3056-21PK-Backend22:55:092
1947614992114,5cyclictest2012988-21sshd22:12:161
1947614992113,6cyclictest1997026-21python321:25:221
1947614992111,8cyclictest2014985-21unin-run22:20:011
194761499210,20cyclictest0-21swapper/122:30:571
194761499210,20cyclictest0-21swapper/120:15:441
1947612992119,1cyclictest3056-21PK-Backend22:25:290
1947612992119,1cyclictest0-21swapper/023:00:230
1947612992119,1cyclictest0-21swapper/022:52:400
1947612992119,1cyclictest0-21swapper/022:00:200
1947612992119,1cyclictest0-21swapper/021:56:290
1947612992119,1cyclictest0-21swapper/020:30:520
1947612992119,1cyclictest0-21swapper/020:26:460
1947612992119,1cyclictest0-21swapper/020:22:060
1947612992119,1cyclictest0-21swapper/019:38:470
1947612992119,1cyclictest0-21swapper/019:31:300
1947612992119,1cyclictest0-21swapper/019:11:090
1947612992113,6cyclictest436-20systemd-journal19:19:590
1947612992112,7cyclictest2061689-21python300:30:200
194761299210,20cyclictest0-21swapper/023:10:040
194761299210,20cyclictest0-21swapper/021:00:180
194761299210,20cyclictest0-21swapper/020:14:080
194761299210,20cyclictest0-21swapper/020:08:210
139099210,14rtkit-daemon3056-21PK-Backend23:09:413
139099210,14rtkit-daemon0-21swapper/200:37:242
139099210,13rtkit-daemon0-21swapper/300:35:443
139099210,13rtkit-daemon0-21swapper/222:35:502
1947622992017,2cyclictest3056-21PK-Backend20:37:473
1947622992017,2cyclictest3056-21PK-Backend00:03:023
1947622992014,4cyclictest886-21NetworkManager20:05:003
194762299200,19cyclictest3056-21PK-Backend22:18:253
1947618992018,1cyclictest3056-21PK-Backend19:25:072
1947618992018,1cyclictest0-21swapper/223:30:382
1947618992018,1cyclictest0-21swapper/223:06:112
1947618992018,1cyclictest0-21swapper/222:21:222
1947618992018,1cyclictest0-21swapper/221:42:142
1947618992018,1cyclictest0-21swapper/221:19:162
1947618992018,1cyclictest0-21swapper/221:14:172
1947618992018,1cyclictest0-21swapper/220:46:592
1947618992018,1cyclictest0-21swapper/219:55:422
1947618992018,1cyclictest0-21swapper/219:55:412
1947618992018,1cyclictest0-21swapper/219:52:312
1947618992018,1cyclictest0-21swapper/219:35:552
1947618992018,1cyclictest0-21swapper/219:32:572
1947618992018,1cyclictest0-21swapper/200:34:252
1947618992012,6cyclictest1990673-21cron21:10:002
194761899200,1cyclictest3056-21PK-Backend22:17:292
194761899200,19cyclictest0-21swapper/222:49:542
194761899200,19cyclictest0-21swapper/219:22:572
194761899200,19cyclictest0-21swapper/219:15:542
194761899200,19cyclictest0-21swapper/219:11:352
194761899200,19cyclictest0-21swapper/200:28:442
194761899200,19cyclictest0-21swapper/200:11:202
1947614992014,4cyclictest26-21ksoftirqd/123:20:121
1947614992013,5cyclictest2007112-21cat21:55:161
194761499200,19cyclictest0-21swapper/120:39:031
194761499200,19cyclictest0-21swapper/119:41:231
1947612992018,1cyclictest0-21swapper/023:50:460
1947612992018,1cyclictest0-21swapper/022:42:260
1947612992018,1cyclictest0-21swapper/022:35:290
1947612992018,1cyclictest0-21swapper/021:14:040
1947612992018,1cyclictest0-21swapper/000:26:290
1947612992016,3cyclictest1895000-21snapd23:08:430
194761299200,1cyclictest0-21swapper/022:31:020
194761299200,19cyclictest3056-21PK-Backend19:42:320
194761299200,19cyclictest0-21swapper/023:36:460
194761299200,19cyclictest0-21swapper/022:59:020
194761299200,19cyclictest0-21swapper/022:48:480
194761299200,19cyclictest0-21swapper/022:23:330
194761299200,19cyclictest0-21swapper/021:27:090
194761299200,19cyclictest0-21swapper/020:45:520
194761299200,19cyclictest0-21swapper/020:01:240
139099200,14rtkit-daemon0-21swapper/200:17:432
1947622991917,1cyclictest3056-21PK-Backend23:24:233
1947622991917,1cyclictest3056-21PK-Backend23:17:543
1947622991917,1cyclictest3056-21PK-Backend00:30:063
1947622991917,1cyclictest0-21swapper/323:52:283
1947622991916,2cyclictest3056-21PK-Backend21:32:013
1947622991916,2cyclictest0-21swapper/319:55:223
1947622991916,2cyclictest0-21swapper/319:55:223
1947622991912,5cyclictest1948456-21sed19:10:173
194762299190,1cyclictest3056-21PK-Backend21:00:593
1947618991917,1cyclictest0-21swapper/222:51:162
1947618991917,1cyclictest0-21swapper/221:23:422
1947618991916,2cyclictest3056-21PK-Backend23:38:352
1947618991911,6cyclictest2049911-21sshd23:57:162
194761899190,1cyclictest0-21swapper/222:01:012
194761899190,1cyclictest0-21swapper/219:45:352
194761899190,18cyclictest0-21swapper/223:00:572
194761499196,12cyclictest2048145-21sh23:55:001
194761499194,14cyclictest1095-21runrttasks21:06:251
1947614991917,1cyclictest0-21swapper/100:15:271
1947614991915,3cyclictest2021875-21cron22:40:011
1947614991912,6cyclictest2058852-21systemd-run00:25:001
1947614991912,5cyclictest1982337-21ls20:45:141
1947614991912,5cyclictest1958436-21seq19:36:511
1947614991911,6cyclictest2029627-21cat23:00:111
1947614991910,7cyclictest1977905-21users20:30:211
194761499190,1cyclictest0-21swapper/119:25:221
194761499190,18cyclictest0-21swapper/121:00:121
1947612991917,1cyclictest0-21swapper/021:35:260
1947612991917,1cyclictest0-21swapper/019:56:240
1947612991917,1cyclictest0-21swapper/019:56:240
1947612991917,1cyclictest0-21swapper/000:20:140
1947612991916,2cyclictest2035213-21strings23:15:180
1947612991914,3cyclictest2006294-21munin-run21:55:000
194761299190,1cyclictest0-21swapper/023:21:060
194761299190,1cyclictest0-21swapper/021:42:230
194761299190,1cyclictest0-21swapper/020:18:200
194761299190,1cyclictest0-21swapper/019:27:110
194761299190,18cyclictest0-21swapper/021:31:350
1947622991816,1cyclictest3056-21PK-Backend23:45:103
1947622991816,1cyclictest3056-21PK-Backend21:56:083
1947622991816,1cyclictest3056-21PK-Backend20:30:173
1947622991816,1cyclictest0-21swapper/319:22:563
1947622991815,2cyclictest3056-21PK-Backend23:01:313
1947622991815,1cyclictest3056-21PK-Backend00:06:063
1947622991811,5cyclictest2019054-21df22:30:143
1947618991816,1cyclictest0-21swapper/222:41:372
1947618991816,1cyclictest0-21swapper/221:38:232
1947618991816,1cyclictest0-21swapper/220:21:032
1947618991811,5cyclictest1977328-21sed20:30:162
194761899180,1cyclictest0-21swapper/221:02:012
1947614991812,5cyclictest2028379-21sed22:55:221
1947614991811,5cyclictest436-20systemd-journal23:11:181
1947614991811,5cyclictest2003030-21sed21:45:011
1947614991811,5cyclictest1973544-21expr20:20:111
1947614991810,6cyclictest1964827-21awk19:55:141
1947614991810,6cyclictest1964827-21awk19:55:141
194761499180,1cyclictest0-21swapper/120:51:471
194761499180,1cyclictest0-21swapper/120:51:471
194761299189,7cyclictest2013946-21cat22:15:160
1947612991816,1cyclictest0-21swapper/022:14:410
1947612991816,1cyclictest0-21swapper/021:46:150
1947612991816,1cyclictest0-21swapper/021:16:340
1947612991816,1cyclictest0-21swapper/000:10:400
1947612991811,5cyclictest2010539-21wc22:05:160
20080382170,3chrt2008030-21munin-run22:00:002
1947622991715,1cyclictest3056-21PK-Backend20:43:013
1947622991713,3cyclictest1968642-21irqstats20:05:173
1947622991712,3cyclictest2050182-21munin-run00:00:013
194762299171,14cyclictest20273162sleep322:55:033
1947622991711,4cyclictest1953575-21sed19:25:023
194762299170,7cyclictest1960185-21sshd19:42:023
1947618991715,1cyclictest0-21swapper/223:24:352
1947618991715,1cyclictest0-21swapper/220:16:242
1947618991715,1cyclictest0-21swapper/220:11:102
194761499179,7cyclictest2024624-21ntp_states22:45:181
194761499179,6cyclictest2016955-21apt22:25:101
194761499179,6cyclictest1970635-21ntp_states20:10:181
194761499179,6cyclictest1956080-21memory19:30:161
1947614991714,2cyclictest0-21swapper/120:40:111
1947614991712,3cyclictest1962688-21acpi19:50:101
1947614991710,6cyclictest2035126-21memory23:15:171
1947614991710,5cyclictest436-20systemd-journal19:48:521
1947614991710,5cyclictest2045583-21cat23:45:181
1947614991710,5cyclictest2042477-21cat23:35:231
1947614991710,5cyclictest2038393-21cat23:25:131
1947614991710,5cyclictest2001188-21sed21:40:011
1947612991714,2cyclictest0-21swapper/000:05:120
1947612991711,4cyclictest1095-21runrttasks20:50:130
1947612991711,4cyclictest1095-21runrttasks20:50:130
194762299164,11cyclictest3056-21PK-Backend22:10:483
1947622991614,1cyclictest3056-21PK-Backend20:54:373
1947622991614,1cyclictest3056-21PK-Backend20:54:373
194762299161,13cyclictest1996161-21apt21:25:133
1947622991611,3cyclictest1993053-21sh21:15:133
194762299160,15cyclictest3056-21PK-Backend22:52:463
194762299160,15cyclictest3056-21PK-Backend22:39:263
1947618991614,1cyclictest0-21swapper/223:43:152
1947618991612,3cyclictest2052307-21tr00:05:132
1947618991611,3cyclictest1975287-21tr20:25:112
1947618991610,5cyclictest1984189-21basename20:50:162
1947618991610,5cyclictest1984189-21basename20:50:152
194761499169,5cyclictest2059811-21ntpq00:25:201
194761499169,5cyclictest2005148-21ls21:50:111
194761499169,5cyclictest1991188-21cat21:10:131
194761499168,6cyclictest2044675-21cron23:45:001
194761499163,11cyclictest2040265-21wc23:30:151
1947614991612,3cyclictest1949688-21sh19:15:021
1947614991612,2cyclictest1992418-21latency_hist21:15:011
1947614991611,3cyclictest886-21NetworkManager22:10:001
1947614991611,3cyclictest2054739-21rt-features00:10:201
1947614991611,3cyclictest2053194-21timerwakeupswit00:05:241
1947614991611,3cyclictest1976040-21xargs20:25:211
194761499161,11cyclictest2062286-21apt-get00:35:041
1947614991610,4cyclictest2156-21gnome-shell19:21:091
194761299169,6cyclictest1981809-21sed20:45:000
194761299168,6cyclictest1980033-21munin-plugin-st20:40:010
1947612991612,2cyclictest1985225-21latency_hist20:55:010
1947612991611,4cyclictest13-21ksoftirqd/000:15:230
1947622991513,1cyclictest3056-21PK-Backend22:22:583
1947622991513,1cyclictest3056-21PK-Backend00:25:403
1947622991512,2cyclictest3056-21PK-Backend23:34:303
1947622991510,4cyclictest3056-21PK-Backend21:40:563
1947622991510,4cyclictest3056-21PK-Backend20:16:593
194762299150,14cyclictest3056-21PK-Backend22:00:443
194762299150,13cyclictest1962679-21/usr/sbin/munin19:50:163
194761899159,4cyclictest2045064-21cpuspeed_turbos23:45:122
194761899158,5cyclictest2020170-21perf22:35:012
194761899157,7cyclictest978-21snmpd20:40:112
194761499159,5cyclictest2051488-21sed00:00:271
194761499159,5cyclictest2026482-21python322:50:211
194761499159,5cyclictest2008349-21cat22:00:111
194761499159,5cyclictest1985953-21ls20:55:121
194761499159,4cyclictest1994866-21cat21:20:161
1947614991511,3cyclictest2049685-21timerwakeupswit23:55:221
1947614991510,3cyclictest1095-21runrttasks23:05:031
1947614991510,3cyclictest1095-21runrttasks19:10:221
1947612991510,4cyclictest2040490-21ntpq23:30:190
194762299144,9cyclictest3056-21PK-Backend21:55:003
1947622991411,2cyclictest3056-21PK-Backend23:36:003
1947622991411,2cyclictest3056-21PK-Backend20:49:073
1947622991411,2cyclictest3056-21PK-Backend19:46:483
1947622991410,3cyclictest3056-21PK-Backend23:26:303
1947622991410,3cyclictest3056-21PK-Backend22:47:413
1947622991410,3cyclictest3056-21PK-Backend22:44:273
1947622991410,3cyclictest3056-21PK-Backend21:45:053
194761499146,6cyclictest1969461-21pool-udisksd20:08:031
1947614991410,3cyclictest1095-21runrttasks21:32:241
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional