You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-19 - 05:33

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: ondemand
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot4s.osadl.org (updated Sun Apr 19, 2026 00:44:14)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
13739162770,6sleep10-21swapper/123:06:221
12025822720,3chrt75-21kworker/3:1-events20:16:173
1134670996742,21cyclictest1409988-21latency_hist23:45:000
1134670996742,21cyclictest1409988-21latency_hist23:45:000
14046972640,4sleep3481ktimers/323:36:303
11851232620,4sleep00-21swapper/019:56:200
11839862600,15chrt75-21kworker/3:1+events19:56:193
1134677995834,9cyclictest1241020-21/usr/sbin/munin20:55:002
14364992570,3sleep0201rcuc/000:11:180
1134670995227,8cyclictest280-21systemd-journal20:15:010
1134670995141,8cyclictest1311578-21diskstats22:06:140
14592542500,5chrt0-21swapper/300:31:313
1134675995032,15cyclictest1271269-21latency_hist21:25:011
1134675994935,11cyclictest1226366-21apt-get20:41:121
1134677994836,8cyclictest1226019-21awk20:40:002
1134675994825,20cyclictest1423917-21ntpq23:56:241
1134670994831,15cyclictest1162148-21fschecks_count19:36:150
1134675994726,19cyclictest1200209-21ntpq20:11:241
1134670994728,3cyclictest1405040-21munin-run23:40:000
1134670994728,2cyclictest1311078-21latency_hist22:05:010
1134675994629,15cyclictest1159073-21kthreadcore19:31:211
1134675994537,5cyclictest1405033-21latency_hist23:40:011
1134675994528,15cyclictest1326849-21irqrtprio22:21:171
12954972440,5sleep1291ktimers/121:46:281
12954972440,5sleep1291ktimers/121:46:281
1134675994438,4cyclictest1156854-21kworker/u8:1+flush-8:019:51:201
1134675994425,6cyclictest1256595-21aten2_r5power_v21:11:141
1134670994424,5cyclictest1375056-21unixbench_multi23:06:310
1134670994334,7cyclictest280-21systemd-journal00:24:590
1134670994325,5cyclictest1241018-21latency_hist20:55:010
1134670994325,4cyclictest1215516-21grep20:26:280
1134670994324,6cyclictest280-21systemd-journal20:10:010
1134670994324,3cyclictest1276057-21unixbench_multi21:26:290
1134670994323,4cyclictest1366769-21kthreadcore23:01:180
1134670994320,9cyclictest1286867-21diskstats21:41:140
1134677994233,6cyclictest1365264-21apt-get22:59:592
1134675994234,6cyclictest1231295-21apt-get20:46:091
1134675994224,3cyclictest1246012-21apt-key21:00:001
1134675994223,3cyclictest1191203-21cron20:05:011
1134670994224,4cyclictest1171970-21cpuspeed_turbos19:46:130
1134670994223,3cyclictest1301640-21cpuspeed_turbos21:56:120
1134670994221,4cyclictest1216465-21cpuspeed_turbos20:31:130
1134675994123,2cyclictest0-21swapper/120:00:011
1134670994132,7cyclictest1266900-21diskstats21:21:120
1134670994132,6cyclictest1459607-21latency_hist00:35:010
1134670994123,4cyclictest1296750-21fschecks_count21:51:150
1134670994123,3cyclictest1395789-21fschecks_time23:31:150
1134675994033,5cyclictest1380622-21/usr/sbin/munin23:16:221
1134675994029,9cyclictest280-21systemd-journal21:57:501
1134675994019,7cyclictest1226007-21latency_hist20:40:001
1134670994024,4cyclictest1201638-21cpuspeed_turbos20:16:110
1134670994024,3cyclictest1390806-21fschecks_count23:26:150
1134670994021,4cyclictest1415541-21fschecks_count23:51:120
1134675993922,15cyclictest1419842-21cron23:55:011
1134675993921,16cyclictest1166107-21unixbench_multi19:36:311
1134670993933,4cyclictest1390180-21idleruntime-cro23:25:000
1134670993929,7cyclictest1235863-21latency_hist20:50:010
1134670993920,4cyclictest1157250-21fschecks_time19:31:150
1134675993820,4cyclictest1311551-21cpuspeed_turbos22:06:131
1134675993819,4cyclictest1395699-21cpuspeed_turbos23:31:131
1134677993728,6cyclictest280-21systemd-journal00:40:012
1134677993725,9cyclictest1088190-21kworker/u8:2+flush-8:019:20:002
1134675993719,7cyclictest1261695-21apt-get21:16:101
1134675993719,4cyclictest1325410-21unixbench_multi22:16:281
1134670993730,5cyclictest1375822-21aten2_r5power_p23:11:120
12534642360,2sleep3471rcuc/321:06:173
1134675993631,4cyclictest1156854-21kworker/u8:1+flush-8:020:51:181
1134675993629,5cyclictest1459944-21apt-get00:36:101
1134675993620,3cyclictest1296708-21cpuspeed_turbos21:51:131
1134675993618,4cyclictest880-21mta-sts-daemon19:21:211
1134675993618,4cyclictest1346106-21cpuspeed_turbos22:41:131
1134675993617,3cyclictest1316458-21cpuspeed_turbos22:11:141
1134670993631,3cyclictest1331373-21fschecks_time22:26:150
1134670993629,5cyclictest280-21systemd-journal19:13:280
1134670993629,5cyclictest1251515-21apt-get21:06:070
1134670993626,8cyclictest1246537-21munin-node21:01:110
1134677993525,8cyclictest1365264-21apt-get23:00:012
1134677993427,5cyclictest1191216-21cat20:05:012
1134675993426,6cyclictest1395143-21latency_hist23:30:001
1134677993328,3cyclictest1215518-21unixbench_multi20:26:282
1134675993327,4cyclictest1375793-21wget23:11:121
1134670993328,4cyclictest1256372-21kworker/u8:4+flush-8:000:06:210
1134670993328,3cyclictest1277469-21irqrtprio21:31:160
1134670993327,5cyclictest989211-21kworker/u8:3+flush-8:020:41:290
1134670993327,4cyclictest0-21swapper/022:20:000
1134670993325,6cyclictest1265429-21ntpq21:16:230
1134670993224,6cyclictest1260776-21grep21:11:320
113467099322,28cyclictest1124799-21kworker/0:0+events19:21:210
11343282324,9sleep20-21swapper/219:07:512
1134677993126,3cyclictest1139204-21unixbench_multi19:11:312
1134675993127,2cyclictest0-21swapper/123:00:001
1134675993123,6cyclictest280-21systemd-journal21:30:001
1134675993120,8cyclictest280-21systemd-journal23:45:511
1134670993124,5cyclictest1365247-21cron23:00:000
1134670993124,5cyclictest1210977-21latency_hist20:25:010
1134670993121,7cyclictest0-21swapper/019:41:210
1134670993121,7cyclictest0-21swapper/019:41:210
1134675993025,3cyclictest1370468-21systemd-run23:05:001
1134675993025,3cyclictest1306901-21irqcore22:01:171
113467099309,19cyclictest1181346-21kworker/0:2+events20:35:370
1134670993025,4cyclictest989210-21kworker/u8:1+flush-8:019:16:200
1134670993024,4cyclictest1256372-21kworker/u8:4+flush-8:022:51:210
1134670993023,5cyclictest280-21systemd-journal22:22:510
11337742301,1sleep00-21swapper/019:06:290
1134677992924,3cyclictest1311441-21aten2_r5power_v22:06:132
1134677992924,3cyclictest1166470-21idleruntime-cro19:40:002
1134677992919,8cyclictest1390491-21munin-node23:26:102
1134675992923,4cyclictest280-21systemd-journal00:03:511
1134670992925,2cyclictest0-21swapper/022:40:000
1134670992924,4cyclictest1256372-21kworker/u8:4+flush-8:022:41:220
1134670992923,4cyclictest1440525-21fschecks_time00:16:140
113467099292,26cyclictest1385264-21kworker/0:2+events23:46:170
12481362280,17sleep375-21kworker/3:1+events21:01:203
1134677992824,2cyclictest1341012-21aten2_r5power_p22:36:102
1134677992823,3cyclictest1376026-21fschecks_count23:11:142
1134677992823,3cyclictest1206452-21aten2_r5power_e20:21:122
1134677992822,4cyclictest1336222-21cpuspeed_turbos22:31:122
1134675992823,3cyclictest1167400-21irqcore19:41:171
1134675992823,3cyclictest1167400-21irqcore19:41:171
1134675992822,4cyclictest1331091-21aten2_r5power_c22:26:121
1134675992822,4cyclictest1281686-21apt-get21:36:101
1134675992822,4cyclictest1206039-21idleruntime-cro20:20:001
1134675992821,5cyclictest1145549-21ntpq19:16:221
1134675992817,9cyclictest280-21systemd-journal21:35:001
113467099284,22cyclictest1330745-21kworker/0:2+events22:46:410
1134670992823,3cyclictest1291784-21cpuspeed_turbos21:46:130
1134670992823,3cyclictest1291784-21cpuspeed_turbos21:46:130
1134670992823,3cyclictest1241664-21fschecks_time20:56:140
1134670992821,5cyclictest1380999-21fschecks_time23:16:150
1134670992817,8cyclictest280-21systemd-journal22:35:000
1134670992810,17cyclictest1124799-21kworker/0:0+events19:26:250
1134677992723,3cyclictest1444467-21unixbench_singl00:16:292
1134677992722,3cyclictest1226479-21aten2_r5power_v20:41:142
1134677992722,2cyclictest1281770-21aten2_r5power_p21:36:122
1134675992722,3cyclictest1356314-21irqcore22:51:171
1134675992721,4cyclictest1439267-21unixbench_multi00:11:291
1134675992721,4cyclictest1251171-21cron21:05:001
1134675992721,4cyclictest1135226-21fschecks_time19:11:141
1134675992720,5cyclictest280-21systemd-journal22:31:511
113467099272,19cyclictest1385264-21kworker/0:2+events23:55:150
1134670992721,4cyclictest1450448-21fschecks_time00:26:140
1134677992622,3cyclictest1266764-21aten2_r5power_v21:21:122
1134677992622,2cyclictest1310669-21unixbench_singl22:01:302
1134677992621,3cyclictest1429466-21unixbench_singl00:01:332
1134677992621,3cyclictest1330339-21unixbench_multi22:21:302
1134677992621,3cyclictest1295820-21unixbench_singl21:46:302
1134677992621,3cyclictest1295820-21unixbench_singl21:46:302
1134677992621,3cyclictest1176029-21unixbench_singl19:46:302
1134677992618,6cyclictest1419184-21grep23:51:252
1134675992620,5cyclictest1256372-21kworker/u8:4+flush-8:021:41:241
1134675992620,4cyclictest1171831-21aten2_r5power_p19:46:111
1134675992616,7cyclictest1444817-21latency_hist00:20:011
1134670992610,14cyclictest1181346-21kworker/0:2+events21:36:260
12952402250,3sleep3451irq_work/321:46:223
12952402250,3sleep3451irq_work/321:46:223
1134677992521,3cyclictest1256834-21fschecks_time21:11:162
1134677992520,3cyclictest1286967-21fschecks_time21:41:142
1134677992519,4cyclictest1375029-21unixbench_multi23:06:322
1134675992516,7cyclictest280-21systemd-journal23:25:001
113467599250,8cyclictest1459258-21unixbench_multi00:31:311
113467099259,14cyclictest1385264-21kworker/0:2+events00:01:210
1134670992521,2cyclictest1188994-21runrttasks20:01:180
1134670992520,3cyclictest1460294-21fschecks_time00:36:150
1134670992519,4cyclictest989211-21kworker/u8:3+flush-8:019:51:140
113467599243,7cyclictest421rcuog/222:46:221
1134675992420,2cyclictest0-21swapper/121:06:081
1134675992420,2cyclictest0-21swapper/100:25:001
1134675992419,3cyclictest1152173-21cpuspeed_turbos19:26:131
1134675992419,3cyclictest0-21swapper/122:36:231
1134675992418,4cyclictest1450321-21cpuspeed_turbos00:26:121
113467099248,15cyclictest1315945-21kworker/0:0+events22:11:270
12651872230,4sleep3481ktimers/321:16:213
12404322230,5sleep3481ktimers/320:51:293
113467799232,15cyclictest1410320-21apt-get23:46:112
1134677992319,2cyclictest1439304-21unixbench_singl00:11:282
1134677992314,7cyclictest1355990-21fschecks_count22:51:142
1134677992314,6cyclictest1390315-21cat23:25:012
1134675992319,2cyclictest0-21swapper/120:31:271
1134675992319,2cyclictest0-21swapper/120:21:151
1134675992319,2cyclictest0-21swapper/120:06:161
1134675992318,3cyclictest1430855-21irqrtprio00:06:181
13842922220,3sleep21384293-21needreboot23:16:222
1134677992218,2cyclictest0-21swapper/222:16:232
1134677992217,3cyclictest1395740-21fschecks_count23:31:142
1134677992216,4cyclictest1346101-21cpuspeed_turbos22:41:132
1134675992218,2cyclictest0-21swapper/123:41:161
1134675992218,2cyclictest0-21swapper/123:41:151
1134675992216,4cyclictest1213856-21kthreadcore20:26:181
11336762224,12sleep10-21swapper/119:06:261
12068002210,2sleep375-21kworker/3:1+mm_percpu_wq20:21:153
113467799218,3cyclictest13174152kthreadcore22:11:192
113467799212,13cyclictest1151940-21apt-get19:26:122
1134677992118,2cyclictest1180916-21unixbench_multi19:51:302
1134677992116,3cyclictest1205652-21unixbench_multi20:16:292
113467799210,4cyclictest1453817-21ntpq00:26:232
113467799206,4cyclictest0-21swapper/200:06:272
113467799202,12cyclictest1261130-21apt-get21:15:032
1134677992016,3cyclictest0-21swapper/220:50:012
1134677992016,2cyclictest1-21systemd22:00:012
1134677992016,2cyclictest0-21swapper/221:05:002
1134677992016,2cyclictest0-21swapper/220:35:012
11341172203,9sleep30-21swapper/319:06:323
1134677991915,2cyclictest0-21swapper/221:30:002
1134677991914,3cyclictest1280699-21mailstats21:31:262
1134677991913,4cyclictest1459270-21unixbench_singl00:31:312
1134677991814,2cyclictest0-21swapper/220:06:182
1134677991811,4cyclictest0-21swapper/223:41:162
1134677991811,4cyclictest0-21swapper/223:41:162
113467799181,12cyclictest1445163-21apt-get00:21:102
113467799181,12cyclictest1150791-21ntpq19:21:242
113467799180,6cyclictest11851282cut19:56:202
1134677991710,5cyclictest1243441-21tr20:56:192
113467799168,6cyclictest1197786-21tr20:11:192
1134677991613,2cyclictest0-21swapper/223:56:282
1134677991613,2cyclictest0-21swapper/222:48:312
1134677991612,2cyclictest1156932-21aten2_r5power_c19:31:122
1134677991611,3cyclictest0-21swapper/223:36:192
1134677991611,3cyclictest0-21swapper/222:26:202
1134677991611,3cyclictest0-21swapper/221:06:182
1134677991611,3cyclictest0-21swapper/219:41:302
1134677991611,3cyclictest0-21swapper/219:41:302
13657332152,8sleep30-21swapper/323:00:033
113467799157,6cyclictest1296425-21seq21:50:452
14405852132,6sleep30-21swapper/300:16:153
12450882132,7sleep30-21swapper/320:56:233
12261612132,6sleep30-21swapper/320:40:013
11705312134,5sleep30-21swapper/319:41:233
11705312134,5sleep30-21swapper/319:41:233
14604382122,6sleep30-21swapper/300:36:183
14392262122,6sleep30-21swapper/300:11:303
14341312122,6sleep30-21swapper/300:06:323
14248212122,5sleep30-21swapper/300:00:023
14190532123,5sleep30-21swapper/323:51:233
14143982122,6sleep30-21swapper/323:46:303
14054032122,6sleep30-21swapper/323:41:123
14054032122,6sleep30-21swapper/323:41:113
13852692122,5sleep30-21swapper/323:20:013
13795502122,6sleep30-21swapper/323:11:263
13711562122,6sleep30-21swapper/323:06:173
13644982122,6sleep30-21swapper/322:56:263
13506442122,6sleep30-21swapper/322:45:023
13453432122,6sleep30-21swapper/322:37:523
13367492122,6sleep30-21swapper/322:31:183
13310632122,6sleep30-21swapper/322:26:113
13297182122,6sleep30-21swapper/322:21:223
13214352122,6sleep30-21swapper/322:16:133
13162142124,5sleep30-21swapper/322:11:113
13013652122,6sleep30-21swapper/321:55:503
12982102122,6sleep30-21swapper/321:51:213
12863052122,5sleep30-21swapper/321:40:013
12816902122,6sleep30-21swapper/321:36:123
12766382122,5sleep30-21swapper/321:30:023
12723102120,2sleep3471rcuc/321:26:163
12601222123,6sleep30-21swapper/321:11:243
12317492123,5sleep30-21swapper/320:46:163
12165872122,6sleep30-21swapper/320:31:153
12151702122,6sleep30-21swapper/320:26:263
11874952122,6sleep30-21swapper/320:01:183
11803892122,6sleep30-21swapper/319:51:223
11717622122,6sleep30-21swapper/319:46:133
11618742122,6sleep30-21swapper/319:36:143
11518272122,6sleep30-21swapper/319:25:023
11462872122,6sleep30-21swapper/319:18:323
11389242122,6sleep30-21swapper/319:11:303
14498282111,6sleep30-21swapper/300:25:023
14492412112,5sleep30-21swapper/300:21:293
14246532112,6sleep30-21swapper/323:58:503
13954802112,5sleep30-21swapper/323:31:123
13907662112,5sleep30-21swapper/323:26:153
13844772112,5sleep30-21swapper/323:16:273
13556742112,5sleep30-21swapper/322:50:523
13496212112,5sleep30-21swapper/322:41:253
13118012112,5sleep30-21swapper/322:06:183
13102642112,5sleep30-21swapper/322:01:283
12670202112,5sleep30-21swapper/321:21:153
12256102112,5sleep30-21swapper/320:36:293
11965092112,5sleep30-21swapper/320:11:133
11916382112,5sleep30-21swapper/320:06:153
11568382112,6sleep30-21swapper/319:30:443
11469092112,5sleep30-21swapper/319:20:163
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional