You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-21 - 21:43

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: ondemand
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack5slot4s.osadl.org (updated Sat Mar 21, 2026 12:44:14)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
114456422070,2sleep3471rcuc/309:01:393
112181221920,9sleep30-21swapper/308:40:003
13313812870,4chrt0-21swapper/012:11:360
11739512850,3sleep375-21kworker/3:1-events09:31:413
13176212570,5sleep30-21swapper/311:56:393
1030751995512,24cyclictest589-21runrttasks11:06:310
1030751995445,6cyclictest1340329-21latency_hist12:20:010
1030751995445,6cyclictest1221005-21latency_hist10:20:010
1030753995346,4cyclictest1111943-21latency_hist08:30:001
1030753995242,7cyclictest1092220-21latency_hist08:10:011
1030753995141,8cyclictest1206537-21/usr/sbin/munin10:06:491
1030751995029,18cyclictest1051518-21sshd07:26:410
1030751994830,16cyclictest280-21systemd-journal11:19:130
1030751994634,8cyclictest280-21systemd-journal09:00:130
1030753994540,3cyclictest1052638-21kworker/u8:3+events_unbound07:51:381
1030751994524,17cyclictest1067274-21latency_hist07:45:010
1030757994435,7cyclictest280-21systemd-journal11:12:132
1030753994434,7cyclictest1186165-21latency_hist09:45:001
103075199441,27cyclictest1072263-21munin-run07:50:000
1030751994334,6cyclictest1171292-21latency_hist09:30:010
1030753994236,4cyclictest1052638-21kworker/u8:3+flush-8:007:36:491
1030753994230,8cyclictest280-21systemd-journal08:47:081
1030751994132,6cyclictest1157008-21diskstats09:16:330
1030753994029,9cyclictest1319530-21ntpq11:56:441
1030751994011,16cyclictest1043286-21idleruntime07:21:360
1030753993934,3cyclictest1030933-21kworker/u8:0+flush-8:008:41:491
1030751993925,12cyclictest1226297-21apt-key10:25:010
103075199389,17cyclictest589-21runrttasks12:35:590
103075199386,23cyclictest0-21swapper/011:59:480
103075199373,26cyclictest0-21swapper/011:41:310
1030753993626,7cyclictest1295609-21latency_hist11:35:011
103075199368,15cyclictest589-21runrttasks09:45:290
1030751993627,7cyclictest1166540-21kworker/u8:4+flush-8:009:36:450
1030751993617,17cyclictest0-21swapper/008:51:380
1030751993615,18cyclictest1216047-21latency_hist10:15:010
1030753993526,7cyclictest280-21systemd-journal11:48:141
1030753993526,6cyclictest1255858-21latency_hist10:55:011
1030751993518,15cyclictest1344971-21unixbench_singl12:21:480
1030751993517,16cyclictest1151117-21unixbench_multi09:06:490
1030753993425,6cyclictest280-21systemd-journal07:47:141
103075199347,21cyclictest141ktimers/011:51:300
103075199343,27cyclictest0-21swapper/009:11:310
1030751993425,7cyclictest1031281-21diskstats07:11:330
1030751993416,16cyclictest1310099-21unixbench_multi11:46:490
1030751993414,18cyclictest1101276-21ntpq08:16:430
103075199334,2cyclictest0-21swapper/008:46:320
103075199333,3cyclictest0-21swapper/007:36:320
103075199333,2cyclictest0-21swapper/007:51:300
103075199333,26cyclictest0-21swapper/011:26:300
1030751993324,6cyclictest1175917-21grep09:31:510
103075199332,3cyclictest0-21swapper/008:21:290
103075199332,2cyclictest0-21swapper/009:57:490
103075199332,27cyclictest0-21swapper/009:41:300
103075199331,9cyclictest1041710-21ntpq07:16:440
1030751993318,13cyclictest1108765-21kthreadcore08:26:380
1030751993318,13cyclictest0-21swapper/009:25:000
1030751993316,15cyclictest1260021-21ntp_offset10:56:450
1030751993315,16cyclictest0-21swapper/011:36:330
1030757993225,5cyclictest280-21systemd-journal10:55:132
1030753993228,3cyclictest1141800-21kworker/u8:1+flush-8:009:31:491
1030753993226,4cyclictest1300794-21kworker/u8:3+events_unbound11:45:011
103075199324,22cyclictest0-21swapper/010:41:290
103075199323,3cyclictest0-21swapper/010:51:300
103075199323,2cyclictest0-21swapper/011:31:310
103075199323,2cyclictest0-21swapper/010:26:320
103075199323,2cyclictest0-21swapper/008:06:300
103075199323,2cyclictest0-21swapper/008:01:320
103075199323,25cyclictest0-21swapper/008:11:300
1030751993225,5cyclictest280-21systemd-journal10:32:200
103075199322,3cyclictest0-21swapper/012:06:300
103075199322,3cyclictest0-21swapper/010:36:300
1030751993221,8cyclictest1078013-21diskstats07:56:350
1030751993211,19cyclictest1121001-21ntpq08:36:420
10304712323,9sleep20-21swapper/207:08:272
10614682310,3sleep375-21kworker/3:1+mm_percpu_wq07:36:443
1030753993126,3cyclictest1196283-21kworker/u8:2+flush-8:010:56:401
1030753993125,4cyclictest206391-21kworker/u8:2+events_unbound08:25:001
1030753993125,4cyclictest1196283-21kworker/u8:2+flush-8:011:06:391
1030753993124,5cyclictest1141800-21kworker/u8:1+flush-8:009:06:341
1030753993122,7cyclictest1250930-21latency_hist10:50:011
103075199313,3cyclictest0-21swapper/008:41:310
103075199313,2cyclictest0-21swapper/012:26:320
103075199313,2cyclictest0-21swapper/012:26:320
103075199313,2cyclictest0-21swapper/007:31:310
103075199313,1cyclictest0-21swapper/010:46:300
1030751993128,2cyclictest1211113-21idleruntime-cro10:10:000
103075199312,3cyclictest0-21swapper/012:31:300
103075199311,9cyclictest1112701-21/usr/sbin/munin08:31:370
103075199311,6cyclictest1030752-21cat07:10:010
1030757993026,2cyclictest1061986-21unixbench_singl07:36:512
1030753993025,4cyclictest1141800-21kworker/u8:1+flush-8:010:11:441
1030753993022,6cyclictest1196283-21kworker/u8:2+events_unbound12:40:011
1030753993020,8cyclictest1220213-21ntpq10:16:441
103075199304,21cyclictest0-21swapper/011:11:300
103075199303,3cyclictest0-21swapper/008:56:310
103075199302,2cyclictest0-21swapper/009:51:310
103075199301,8cyclictest1281571-21/usr/sbin/munin11:21:350
103075199300,8cyclictest1201854-21/usr/sbin/munin10:01:350
1030753992925,2cyclictest1042500-21cron07:20:001
1030753992924,4cyclictest1166540-21kworker/u8:4+flush-8:009:36:351
1030753992923,4cyclictest1044748-21kthreadcore07:21:401
1030751992919,8cyclictest1264941-21ntpq11:01:450
103075199291,8cyclictest1324686-21ntpq12:01:430
1030757992824,3cyclictest1126772-21idleruntime-cro08:45:002
1030757992818,7cyclictest280-21systemd-journal12:10:002
1030753992824,2cyclictest1191136-21awk09:50:011
1030753992823,3cyclictest206391-21kworker/u8:2+flush-8:007:31:501
1030753992823,3cyclictest1096404-21gcc08:11:461
1030753992821,5cyclictest1270952-21kworker/u8:0+events_unbound11:40:001
1030757992724,2cyclictest1246388-21aten2_r5power_c10:46:312
1030757992723,2cyclictest1191729-21aten2_r5power_e09:51:332
1030757992723,2cyclictest0-21swapper/208:50:002
1030753992722,4cyclictest1141800-21kworker/u8:1+flush-8:009:51:351
1030753992722,3cyclictest1312977-21kthreadcore11:51:391
1030757992623,2cyclictest1156396-21latency_hist09:15:002
1030757992621,4cyclictest998988-21kworker/u8:1+flush-8:007:21:362
1030757992620,4cyclictest280-21systemd-journal09:05:002
1030753992621,3cyclictest1330434-21systemd-run12:10:001
1030753992620,5cyclictest1196283-21kworker/u8:2+flush-8:010:26:361
1030753992619,5cyclictest1300794-21kworker/u8:3+flush-8:012:11:371
1030753992618,6cyclictest1166539-21kworker/u8:2+flush-8:009:26:481
1030753992612,4cyclictest0-21swapper/109:56:371
11007882250,4sleep2371rcuc/208:16:402
1030757992522,2cyclictest1180882-21unixbench_singl09:36:512
1030757992521,2cyclictest1350640-21aten2_r5power_e12:31:332
1030753992521,3cyclictest1270952-21kworker/u8:0+events_unbound12:05:011
1030753992520,4cyclictest1030933-21kworker/u8:0+flush-8:007:56:491
1030753992519,3cyclictest0-21swapper/112:21:451
1030753992514,7cyclictest0-21swapper/111:01:331
1030757992421,2cyclictest1296191-21cpuspeed_turbos11:36:312
1030757992419,3cyclictest1265752-21sh11:05:002
1030757992418,4cyclictest371rcuc/209:26:322
1030757992416,6cyclictest280-21systemd-journal11:15:132
1030753992420,2cyclictest1231574-21cut10:31:341
1030753992419,3cyclictest1240302-21collect210:36:451
1030753992419,3cyclictest0-21swapper/111:16:341
1030753992419,2cyclictest0-21swapper/110:21:451
1030753992418,5cyclictest1141800-21kworker/u8:1+flush-8:009:21:381
1030753992417,5cyclictest1350581-21apt-get12:31:301
1030753992416,6cyclictest280-21systemd-journal08:56:141
1030753992416,6cyclictest206391-21kworker/u8:2+flush-8:008:31:461
1030753992416,6cyclictest1300794-21kworker/u8:3+flush-8:012:26:451
1030753992416,6cyclictest1300794-21kworker/u8:3+flush-8:012:26:451
1030753992416,6cyclictest1196283-21kworker/u8:2+flush-8:010:41:431
1030753992414,8cyclictest1051539-21ntpq07:26:421
1030757992320,2cyclictest1330805-21aten2_r5power_c12:11:312
1030757992320,2cyclictest1320972-21aten2_r5power_c12:01:322
1030757992320,2cyclictest1067927-21fschecks_count07:46:342
1030757992319,2cyclictest1231829-21fschecks_time10:31:362
1030757992315,6cyclictest1255513-21unixbench_multi10:51:502
1030753992319,3cyclictest1141800-21kworker/u8:1+flush-8:009:11:481
1030753992318,4cyclictest1099657-21kthreadcore08:16:401
1030753992318,3cyclictest1205316-21needreboot10:01:431
1030753992318,3cyclictest0-21swapper/112:20:011
1030753992318,3cyclictest0-21swapper/109:16:501
1030753992318,2cyclictest0-21swapper/111:26:381
1030753992317,4cyclictest206391-21kworker/u8:2+flush-8:007:41:451
1030753992317,4cyclictest1132984-21kthreadcore08:51:391
1030753992317,4cyclictest1030933-21kworker/u8:0+flush-8:008:36:481
1030757992218,2cyclictest1355603-21aten2_r5power_p12:36:332
1030757992218,2cyclictest1286368-21fschecks_count11:26:342
1030757992218,2cyclictest1281412-21cpuspeed_turbos11:21:332
1030757992216,4cyclictest1320012-21unixbench_singl11:56:502
1030753992218,2cyclictest0-21swapper/107:11:351
1030753992217,3cyclictest0-21swapper/108:01:351
1030753992216,4cyclictest1145735-21ntpq09:01:431
103075799217,10cyclictest1-21systemd09:10:012
1030757992118,2cyclictest1205289-21tail10:01:432
1030757992117,2cyclictest1181600-21aten2_r5power_c09:41:312
1030757992113,6cyclictest1230606-21sort10:26:502
1030757992112,7cyclictest1295301-21unixbench_singl11:31:502
1030753992117,3cyclictest401-21dbus-daemon11:25:011
1030753992117,2cyclictest0-21swapper/111:11:371
103075799202,13cyclictest1072804-21apt-get07:51:322
1030757992017,2cyclictest1350226-21idleruntime-cro12:30:002
1030757992017,2cyclictest1350226-21idleruntime-cro12:30:002
1030757992016,2cyclictest1035669-21cron07:15:002
1030757992016,2cyclictest0-21swapper/209:21:462
1030757992015,3cyclictest1116571-21unixbench_singl08:31:512
1030757992012,6cyclictest1141533-21cron09:00:002
1030757992011,7cyclictest1094774-21tr08:11:392
10259992204,10sleep10-21swapper/107:05:371
103075799194,3cyclictest13394892ld12:16:432
1030757991916,2cyclictest1082851-21aten2_r5power_v08:01:332
1030757991916,2cyclictest1052798-21aten2_r5power_e07:31:322
1030757991916,2cyclictest1047856-21aten2_r5power_c07:26:312
1030757991916,2cyclictest0-21swapper/208:36:472
1030757991915,2cyclictest0-21swapper/212:21:332
1030757991915,2cyclictest0-21swapper/210:16:422
1030757991915,2cyclictest0-21swapper/210:11:502
1030757991915,2cyclictest0-21swapper/209:56:452
1030757991910,7cyclictest1225624-21unixbench_multi10:21:502
1030757991814,2cyclictest0-21swapper/210:06:482
1030757991814,2cyclictest0-21swapper/208:10:002
1030757991813,3cyclictest1186637-21aten2_r5power_v09:46:312
1030757991813,3cyclictest0-21swapper/211:55:012
1030757991812,4cyclictest0-21swapper/208:51:522
10303582182,9sleep30-21swapper/307:07:003
103075799179,6cyclictest1037748-21sort07:15:052
103075799178,7cyclictest1303274-21cut11:41:402
103075799178,7cyclictest1102750-21fschecks_count08:21:332
103075799171,5cyclictest1078298-21irqcore07:56:392
1030757991713,3cyclictest1161019-21unixbench_singl09:16:492
1030757991712,3cyclictest0-21swapper/210:36:402
1030757991712,3cyclictest0-21swapper/208:26:382
11413552162,5sleep30-21swapper/308:57:433
11220822162,5sleep30-21swapper/308:40:213
103075799168,6cyclictest1268134-21tr11:06:392
103075799168,6cyclictest1242092-21sed10:41:362
103075799162,10cyclictest1309612-21ntpq11:46:422
103075799161,11cyclictest1172053-21idleruntime09:31:372
103075799161,11cyclictest1062679-21apt-get07:41:292
12189502156,5sleep30-21swapper/310:16:403
10823322152,4sleep30-21swapper/307:59:533
13145052142,8sleep30-21swapper/311:51:423
13400942132,9sleep30-21swapper/312:17:103
13053522131,4sleep30-21swapper/311:42:393
12111592132,6sleep30-21swapper/310:10:023
11808232132,6sleep30-21swapper/309:36:513
10723752132,6sleep30-21swapper/307:50:023
10428032132,9sleep30-21swapper/307:20:403
10317602133,6sleep30-21swapper/307:11:373
13558032122,6sleep30-21swapper/312:36:363
13459942123,5sleep30-21swapper/312:26:373
13459942123,5sleep30-21swapper/312:26:373
13441252122,6sleep30-21swapper/312:21:403
13312452122,6sleep30-21swapper/312:11:363
13261832122,6sleep30-21swapper/312:06:353
12968562122,6sleep30-21swapper/311:36:373
12857572122,6sleep30-21swapper/311:25:023
12798302122,6sleep30-21swapper/311:16:433
12754902122,6sleep30-21swapper/311:13:143
12558122124,6sleep30-21swapper/310:54:323
12460012122,5sleep30-21swapper/310:45:013
12401572122,6sleep30-21swapper/310:36:423
12254122122,6sleep30-21swapper/310:21:503
12015382127,4sleep30-21swapper/310:00:593
12002362122,6sleep30-21swapper/309:56:423
11929332122,6sleep30-21swapper/309:51:403
11572492122,6sleep30-21swapper/309:16:373
11465652122,5sleep30-21swapper/309:05:013
11128102122,6sleep30-21swapper/308:31:393
11024832122,6sleep30-21swapper/308:21:313
11018342122,6sleep30-21swapper/308:17:023
10929732122,6sleep30-21swapper/308:11:363
10875102122,5sleep30-21swapper/308:05:023
10530492122,6sleep30-21swapper/307:31:363
10474892122,6sleep30-21swapper/307:25:023
10388452122,6sleep30-21swapper/307:16:393
13512892112,5sleep30-21swapper/312:31:393
13210432112,5sleep30-21swapper/312:01:333
13063802112,5sleep30-21swapper/311:46:363
12913262112,6sleep30-21swapper/311:31:363
12810822113,5sleep30-21swapper/311:20:243
12660322112,5sleep30-21swapper/311:05:473
12611862112,5sleep30-21swapper/311:01:343
12561102113,5sleep30-21swapper/310:55:203
12414312112,5sleep30-21swapper/310:41:323
12353782112,5sleep30-21swapper/310:31:493
12305922112,5sleep30-21swapper/310:26:513
12064512112,6sleep30-21swapper/310:05:243
11864012112,5sleep30-21swapper/309:45:143
11851452112,5sleep30-21swapper/309:41:413
11666192112,6sleep30-21swapper/309:26:093
11615762112,5sleep30-21swapper/309:20:023
11562492112,5sleep30-21swapper/309:13:163
11339592112,5sleep30-21swapper/308:51:413
11273812112,5sleep30-21swapper/308:46:353
11078562112,5sleep30-21swapper/308:26:373
10831762112,5sleep30-21swapper/308:01:373
10675172112,5sleep30-21swapper/307:45:163
10625412112,5sleep30-21swapper/307:40:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional