You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-10 - 22:45

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: ondemand
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot4s.osadl.org (updated Wed Jun 10, 2026 12:44:15)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
127153422400,3sleep3471rcuc/312:10:333
12367032980,2sleep375-21kworker/3:1-events11:35:013
11619852740,3chrt75-21kworker/3:1+events10:20:003
9874672690,2sleep3471rcuc/307:20:473
9890852580,2sleep3451irq_work/307:25:373
9890852580,2sleep3451irq_work/307:25:363
971105995534,18cyclictest1087794-21/usr/sbin/munin09:05:461
971105995234,16cyclictest1206909-21apt-get11:05:281
971103995241,9cyclictest975069-21ntpq07:10:430
12857732500,5sleep20-21swapper/212:20:462
971105994939,8cyclictest1006780-21ntpq07:40:421
971105994738,7cyclictest1155892-21ntpq10:10:411
971105994737,8cyclictest1260453-21ntpq11:55:441
971105994735,9cyclictest1280185-21ntpq12:15:441
971105994626,18cyclictest1704890-21systemd-journal09:17:001
971105994626,18cyclictest1704890-21systemd-journal09:17:001
971105994534,8cyclictest1704890-21systemd-journal11:01:511
971105994530,13cyclictest1290268-21ntpq12:25:411
971103994433,8cyclictest1240593-21ntpq11:35:420
971105994334,7cyclictest1042455-21cp08:20:011
971105994334,7cyclictest1021629-21ntpq07:55:411
971105994334,6cyclictest1176893-21latency_hist10:35:011
971105994321,19cyclictest1236289-21latency_hist11:35:001
971105994224,16cyclictest1086411-21ntpq09:00:411
971103994034,4cyclictest1037605-21kworker/u8:1+flush-8:009:40:470
971103994033,5cyclictest1246714-21/usr/sbin/munin11:45:420
971103994033,4cyclictest988138-21apt-get07:25:290
971103994033,4cyclictest988138-21apt-get07:25:290
971103994032,6cyclictest1231582-21/usr/sbin/munin11:30:440
971103994029,8cyclictest1096636-21ntpq09:10:420
971103993931,5cyclictest1231342-21true11:30:000
971105993830,6cyclictest1201663-21latency_hist11:00:011
971105993830,6cyclictest1201663-21latency_hist11:00:001
971105993823,13cyclictest1097535-21munin-run09:14:591
971105993818,2cyclictest0-21swapper/107:35:481
971105993817,5cyclictest1704890-21systemd-journal09:40:011
971103993829,6cyclictest1002693-21latency_hist07:40:020
9700442383,30sleep30-21swapper/307:06:113
971105993729,6cyclictest1704890-21systemd-journal10:55:001
971105993727,7cyclictest1704890-21systemd-journal08:00:511
971105993720,15cyclictest0-21swapper/109:50:011
971105993719,15cyclictest0-21swapper/110:05:441
971105993626,8cyclictest1126253-21ntpq09:40:391
971105993620,14cyclictest1166983-21ls10:25:011
971105993617,2cyclictest0-21swapper/108:55:001
971105993616,2cyclictest0-21swapper/107:25:401
971105993616,2cyclictest0-21swapper/107:25:401
971103993632,3cyclictest137713-21kworker/u8:2+events_unbound08:00:360
971103993627,7cyclictest1032122-21kworker/u8:0+flush-8:008:10:340
971103993626,8cyclictest1117802-21diskstats09:35:300
971105993528,5cyclictest1031605-21ntpq08:05:441
971105993521,12cyclictest0-21swapper/112:39:591
971105993519,14cyclictest1116903-21unixbench_singl09:30:481
971105993519,14cyclictest0-21swapper/108:55:311
971105993518,15cyclictest987872-21systemd-run07:24:591
971105993517,2cyclictest0-21swapper/108:35:411
971105993517,2cyclictest0-21swapper/108:30:341
971105993517,2cyclictest0-21swapper/108:20:341
971105993517,2cyclictest0-21swapper/107:50:441
971105993516,2cyclictest0-21swapper/111:50:311
971105993516,2cyclictest0-21swapper/109:50:371
971105993515,4cyclictest0-21swapper/112:20:391
971103993526,6cyclictest1704890-21systemd-journal11:00:370
971103993525,8cyclictest1190848-21ntpq10:45:410
9706762353,9sleep10-21swapper/107:07:011
11149072350,5sleep3451irq_work/309:30:373
971105993425,7cyclictest1160866-21ntpq10:15:401
971105993418,2cyclictest0-21swapper/111:45:501
971105993417,2cyclictest0-21swapper/111:20:471
971105993415,7cyclictest1216753-21apt-get11:15:281
971103993429,3cyclictest1087352-21systemd-run09:05:000
971103993427,5cyclictest1255430-21ntpq11:50:420
971105993326,5cyclictest1111971-21grep09:25:491
971105993318,2cyclictest976127-21sed07:15:011
971105993317,2cyclictest0-21swapper/112:10:331
971103993327,4cyclictest1175957-21ntpq10:30:390
971103993326,4cyclictest1016663-21ntpq07:50:430
971103993323,8cyclictest1195817-21ntpq10:50:430
971103993323,8cyclictest1076549-21ntpq08:50:430
97110599329,9cyclictest1137324-21apt-get09:55:291
971105993228,2cyclictest1036412-21needreboot08:10:411
971105993224,6cyclictest1211841-21apt-get11:10:281
971105993218,1cyclictest0-21swapper/107:20:001
971105993216,2cyclictest0-21swapper/110:40:391
971103993228,2cyclictest1197931-21kthreadcore10:55:360
971103993228,2cyclictest1197931-21kthreadcore10:55:350
971103993227,4cyclictest137713-21kworker/u8:2+flush-8:008:40:550
971103993224,6cyclictest1286237-21latency_hist12:25:010
971103993224,6cyclictest1245573-21ntpq11:40:420
971103993123,6cyclictest1081439-21ntpq08:55:400
9708722313,22sleep00-21swapper/007:09:340
971108993025,3cyclictest1057503-21kworker/u8:3+flush-8:009:35:472
971105993018,9cyclictest1704890-21systemd-journal11:27:521
971105993011,6cyclictest1704890-21systemd-journal12:04:591
971103993021,7cyclictest1171032-21ntpq10:25:390
971103993020,8cyclictest1051891-21ntpq08:25:450
971103993019,9cyclictest1266770-21diskstats12:05:320
971103993019,8cyclictest1281045-21apt-get12:20:000
971108992925,3cyclictest1037605-21kworker/u8:1+flush-8:008:15:432
971108992924,3cyclictest1057212-21unixbench_singl08:30:492
971108992923,4cyclictest1241109-21unixbench_singl11:35:502
971108992920,7cyclictest1052694-21cron08:29:592
971105992926,2cyclictest1189271-21kthreadcore10:45:371
971105992921,6cyclictest1007986-21apt-get07:45:281
971105992912,3cyclictest0-21swapper/112:30:351
97110599291,12cyclictest583-21snmpd11:35:301
971103992925,2cyclictest1156850-21systemd-run10:15:000
971103992924,4cyclictest137713-21kworker/u8:2+flush-8:008:49:220
971103992924,3cyclictest1301058-21systemd-run12:40:000
971103992923,3cyclictest982867-21idleruntime-cro07:19:590
971103992920,6cyclictest1102494-21latency_hist09:20:010
971103992920,6cyclictest1102494-21latency_hist09:20:010
971103992913,14cyclictest1704890-21systemd-journal11:56:510
971108992818,8cyclictest1047965-21cp08:25:012
971105992822,4cyclictest137713-21kworker/u8:2+flush-8:008:25:221
971103992823,3cyclictest0-21swapper/008:00:010
971103992822,4cyclictest1042907-21aten2_r5power_c08:20:310
971108992723,2cyclictest1196334-21unixbench_singl10:50:502
971108992722,4cyclictest1191792-21kworker/u8:0+events_unbound11:00:372
971108992722,3cyclictest1261878-21fschecks_count12:00:352
971103992723,3cyclictest923438-21kworker/u8:3+flush-8:007:20:420
971103992720,5cyclictest1037605-21kworker/u8:1+flush-8:008:30:380
971103992717,2cyclictest421rcuog/210:05:500
971108992622,3cyclictest1230999-21kworker/u8:3+flush-8:012:05:422
971108992622,3cyclictest1191792-21kworker/u8:0+flush-8:011:10:382
971108992622,2cyclictest1246826-21aten2_r5power_p11:45:322
971105992621,4cyclictest1147401-21kworker/u8:3+flush-8:010:35:361
971105992621,3cyclictest0-21swapper/111:45:001
971105992618,6cyclictest409-21smartd08:49:471
971105992616,8cyclictest1146101-21ntpq10:00:441
971103992618,6cyclictest1037605-21kworker/u8:1+flush-8:009:45:310
971103992618,6cyclictest1037605-21kworker/u8:1+flush-8:009:05:430
971103992610,8cyclictest1037605-21kworker/u8:1+events_unbound09:50:280
971108992522,2cyclictest1092837-21aten2_r5power_c09:10:292
971105992519,4cyclictest993074-21apt-get07:30:271
97110399252,17cyclictest589-21runrttasks09:25:300
971103992521,3cyclictest1251085-21kworker/u8:0+flush-8:012:31:210
971103992521,3cyclictest1002424-21kworker/u8:1+flush-8:007:40:320
971103992520,3cyclictest1182241-21aten2_r5power_p10:40:300
971103992520,3cyclictest1037826-21cpuspeed08:15:310
971103992513,9cyclictest1266240-21latency_hist12:05:000
97110399251,21cyclictest1116934-21sshd09:30:510
971108992421,2cyclictest971537-21fschecks_count07:10:342
971108992421,2cyclictest1136612-21unixbench_multi09:50:482
971108992421,2cyclictest1083025-21fschecks_time09:00:332
971108992420,3cyclictest1191792-21kworker/u8:0+flush-8:011:20:442
971108992420,3cyclictest1167548-21fschecks_count10:25:322
971108992420,2cyclictest1077914-21cpuspeed_turbos08:55:292
971108992419,4cyclictest1230999-21kworker/u8:3+events_unbound11:50:472
971108992419,3cyclictest1177468-21fschecks_count10:35:332
971103992419,3cyclictest1146549-21unixbench_singl10:00:480
971103992419,3cyclictest0-21swapper/011:20:010
971108992320,2cyclictest988453-21fschecks_time07:25:332
971108992320,2cyclictest988453-21fschecks_time07:25:322
971108992320,2cyclictest1137639-21fschecks_time09:55:332
971108992319,3cyclictest137713-21kworker/u8:2+flush-8:007:55:442
971108992319,2cyclictest1235902-21unixbench_singl11:30:492
971108992319,2cyclictest0-21swapper/207:20:472
971108992317,4cyclictest1116844-21unixbench_multi09:30:482
971108992316,5cyclictest993357-21grep07:30:332
97110899230,21cyclictest1-21systemd12:20:002
97110899230,10cyclictest11668952readlink10:25:012
97110399232,14cyclictest583-21snmpd07:45:280
971103992319,2cyclictest882-21mta-sts-daemon10:21:070
971103992318,3cyclictest1057503-21kworker/u8:3+flush-8:008:35:460
97110399231,19cyclictest589-21runrttasks10:35:310
971108992217,4cyclictest1127167-21idleruntime-cro09:45:002
971108992217,3cyclictest824873-21kworker/u8:1+events_unbound07:15:482
971105992218,2cyclictest1067565-21cron08:44:591
971105992217,3cyclictest0-21swapper/109:20:301
971103992218,3cyclictest8550irq/124-ahci[0000:00:12.0]07:30:300
971103992215,5cyclictest876-21mta-sts-daemon12:26:290
971108992118,2cyclictest1032755-21aten2_r5power_c08:10:312
971108992116,4cyclictest0-21swapper/211:20:002
971105992116,3cyclictest0-21swapper/112:05:381
97110399212,13cyclictest583-21snmpd11:05:440
971103992117,3cyclictest1037605-21kworker/u8:1+flush-8:010:15:310
971103992117,2cyclictest1138045-21irqrtprio09:55:350
971103992117,2cyclictest0-21swapper/011:20:330
971103992115,5cyclictest923438-21kworker/u8:3+flush-8:008:06:570
97110899208,9cyclictest1147215-21apt-get10:05:292
971108992017,2cyclictest1161757-21idleruntime-cro10:19:592
971108992016,3cyclictest1057503-21kworker/u8:3+events_unbound09:10:002
971108992016,2cyclictest1072121-21unixbench_singl08:45:482
971108992014,4cyclictest1067184-21unixbench_singl08:40:512
971108992013,5cyclictest1107465-21seq09:25:002
971105992014,4cyclictest0-21swapper/110:25:381
971103992016,2cyclictest1276109-21cron12:15:000
9661922204,9sleep20-21swapper/207:05:012
11013942200,2sleep20-21swapper/209:15:392
11013942200,2sleep20-21swapper/209:15:382
971108991915,3cyclictest1176694-21kworker/u8:1+flush-8:011:05:422
971108991915,2cyclictest1072871-21aten2_r5power_p08:50:322
971108991914,4cyclictest137713-21kworker/u8:2+flush-8:007:50:372
971108991910,7cyclictest1154055-21tr10:10:392
971108991910,7cyclictest1058968-21tr08:35:352
971103991914,3cyclictest1103715-21kthreadcore09:20:350
971103991913,4cyclictest1215122-21kthreadcore11:10:380
10113452190,2sleep20-21swapper/207:45:382
971108991815,2cyclictest1142593-21fschecks_time10:00:322
971108991813,3cyclictest0-21swapper/208:05:372
971108991714,1cyclictest0-21swapper/210:30:292
971108991713,2cyclictest1297080-21irqrtprio12:35:372
971108991713,2cyclictest0-21swapper/211:25:482
971108991713,2cyclictest0-21swapper/210:45:272
971108991713,2cyclictest0-21swapper/207:40:492
971108991713,2cyclictest0-21swapper/207:35:502
971108991712,3cyclictest0-21swapper/210:55:372
971108991712,3cyclictest0-21swapper/210:55:372
971108991710,5cyclictest1276094-21cron12:15:002
971108991710,5cyclictest1108620-21cut09:25:352
971108991710,5cyclictest0-21swapper/208:00:402
97110899170,15cyclictest1-21systemd11:56:512
97110899163,8cyclictest1182156-21apt-get10:40:302
97110899162,4cyclictest0-21swapper/212:25:222
12810302164,8sleep30-21swapper/312:20:013
97110899157,6cyclictest1292236-21sed12:30:342
971108991512,2cyclictest0-21swapper/209:45:172
97110899151,11cyclictest1241802-21apt-get11:40:302
11469182142,7sleep30-21swapper/310:05:003
11177952143,8sleep30-21swapper/309:35:323
10393102144,6sleep30-21swapper/308:15:383
12536032133,7sleep30-21swapper/311:50:383
12169982132,7sleep30-21swapper/311:15:323
12116042132,6sleep30-21swapper/311:10:023
11471982132,7sleep30-21swapper/310:05:293
11228932132,7sleep30-21swapper/309:40:333
10986992134,5sleep30-21swapper/309:15:373
10986992134,5sleep30-21swapper/309:15:363
10126512132,6sleep30-21swapper/307:50:023
9931612122,6sleep30-21swapper/307:30:323
9782612122,6sleep30-21swapper/307:15:313
12967352122,6sleep30-21swapper/312:35:343
12670382122,6sleep30-21swapper/312:05:353
12624062122,6sleep30-21swapper/312:00:393
12568752122,6sleep30-21swapper/311:55:333
12464732122,6sleep30-21swapper/311:45:023
12422972122,6sleep30-21swapper/311:40:363
12405692122,6sleep30-21swapper/311:35:423
12306732122,6sleep30-21swapper/311:25:473
12260382122,6sleep30-21swapper/311:20:493
11958192122,6sleep30-21swapper/310:50:433
11821282122,6sleep30-21swapper/310:40:293
11809452122,6sleep30-21swapper/310:35:423
11719732122,5sleep30-21swapper/310:30:013
11370702122,6sleep30-21swapper/309:55:023
11360772122,6sleep30-21swapper/309:50:413
11076792122,6sleep30-21swapper/309:25:283
11074072122,6sleep30-21swapper/309:25:003
10933312122,6sleep30-21swapper/309:10:363
10878232122,6sleep30-21swapper/309:05:303
10829192122,6sleep30-21swapper/309:00:323
10627072122,6sleep30-21swapper/308:40:023
10552362123,6sleep30-21swapper/308:30:393
10487542122,6sleep30-21swapper/308:25:363
10444452122,6sleep30-21swapper/308:20:403
10087432122,6sleep30-21swapper/307:45:363
10025242122,6sleep30-21swapper/307:38:053
9753212112,5sleep30-21swapper/307:10:483
12951472112,5sleep30-21swapper/312:30:403
12864552112,5sleep30-21swapper/312:25:023
12815812112,5sleep30-21swapper/312:20:293
12071572112,5sleep30-21swapper/311:05:333
12024012112,5sleep30-21swapper/311:00:343
11967512112,5sleep30-21swapper/310:55:023
11967512112,5sleep30-21swapper/310:55:023
11874852112,6sleep30-21swapper/310:45:343
11672222112,5sleep30-21swapper/310:25:043
11619852112,5sleep30-21swapper/310:20:023
11560262112,5sleep30-21swapper/310:10:473
11301592112,5sleep30-21swapper/309:45:383
10776392111,6sleep30-21swapper/308:55:033
10724902111,5sleep30-21swapper/308:50:013
10683692112,5sleep30-21swapper/308:45:353
10582102112,6sleep30-21swapper/308:35:323
10331492112,5sleep30-21swapper/308:10:353
10277312112,5sleep30-21swapper/308:05:033
10237412112,5sleep30-21swapper/308:00:373
10219052112,5sleep30-21swapper/307:55:473
10028642112,5sleep30-21swapper/307:40:033
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional