You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-15 - 06:24

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: ondemand
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot4s.osadl.org (updated Mon Dec 15, 2025 00:44:13)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
161116321810,2sleep3471rcuc/323:14:083
140467021770,5sleep30-21swapper/319:44:083
16662732560,2chrt575-21kworker/3:2+events00:09:083
14699752560,3sleep3575-21kworker/3:2-events20:49:083
1370083995245,5cyclictest1554620-21latency_hist22:15:010
1370088995043,5cyclictest278-21systemd-journal23:14:172
1370084994942,5cyclictest1370323-21kworker/u8:1+flush-8:019:39:171
1370084994836,8cyclictest1411375-21turbostat.cron19:50:011
1370084994739,5cyclictest277023-21kworker/u8:0+flush-8:019:19:151
1370084994339,3cyclictest1451038-21kworker/u8:2+flush-8:021:04:151
137008399421,39cyclictest1-21systemd21:20:000
1370084994134,5cyclictest1456237-21kworker/u8:0+flush-8:022:09:321
1370084994133,6cyclictest1569958-21kworker/u8:0+flush-8:022:34:171
1370084994133,6cyclictest1569958-21kworker/u8:0+flush-8:022:34:171
1370083994131,7cyclictest1480584-21latency_hist21:00:010
137008399412,33cyclictest1365038-21kworker/0:1+events20:01:080
1370084994032,6cyclictest1470778-21kworker/u8:3+flush-8:022:04:141
136985024027,7sleep10-21swapper/119:09:371
1370084993932,5cyclictest1658999-21kworker/u8:1+flush-8:000:29:151
1370083993929,7cyclictest1391635-21latency_hist19:30:000
1370084993833,3cyclictest1470778-21kworker/u8:3+events_unbound21:54:041
1370084993830,6cyclictest1584787-21/usr/sbin/munin22:49:151
137008399382,30cyclictest1365038-21kworker/0:1+events19:13:550
1370083993731,4cyclictest1461467-21apt-get20:43:590
1370083993727,8cyclictest278-21systemd-journal20:50:010
137008399370,35cyclictest1593638-21sshd22:54:170
1370084993623,9cyclictest1633906-21latency_hist23:35:001
1370083993625,9cyclictest1563884-21ntpq22:24:130
1370084993528,5cyclictest1470778-21kworker/u8:3+flush-8:021:39:421
1370084993528,5cyclictest1451038-21kworker/u8:2+flush-8:020:49:051
1370083993523,10cyclictest1669706-21diskstats00:14:030
1370083993425,7cyclictest278-21systemd-journal22:00:010
1370084993328,4cyclictest277023-21kworker/u8:0+flush-8:019:24:201
1370084993328,3cyclictest1505154-21turbostat.cron21:25:001
1370084993326,5cyclictest277023-21kworker/u8:0+flush-8:019:44:031
1370084993326,5cyclictest1456237-21kworker/u8:0+events_unbound21:29:171
1370084993325,6cyclictest1674347-21/usr/sbin/munin00:19:161
137008399332,30cyclictest1451058-21kworker/0:1+events21:44:070
137008399332,30cyclictest1365038-21kworker/0:1+events20:14:270
137008399332,26cyclictest1431219-21kworker/0:0+events20:15:520
137008399331,30cyclictest1451058-21kworker/0:1+events21:29:160
137008399331,26cyclictest1451058-21kworker/0:1+events22:33:590
137008399331,26cyclictest1451058-21kworker/0:1+events22:33:590
1370083993310,19cyclictest1451058-21kworker/0:1+events21:20:350
13655062334,9sleep20-21swapper/219:08:252
1370084993227,3cyclictest1575209-21fschecks_count22:39:031
1370084993225,5cyclictest1510565-21kworker/u8:1+flush-8:022:14:181
1370084993225,4cyclictest0-21swapper/120:09:061
137008399323,27cyclictest1451058-21kworker/0:1+events23:13:590
137008399322,29cyclictest1365038-21kworker/0:1+events19:47:470
1370084993125,3cyclictest0-21swapper/121:49:101
1370084993123,6cyclictest1549714-21kworker/u8:2+flush-8:023:29:141
137008399313,26cyclictest1365038-21kworker/0:1+events19:38:590
1370083993123,5cyclictest278-21systemd-journal20:20:450
137008399312,27cyclictest1451058-21kworker/0:1+events21:49:150
137008399310,30cyclictest1451058-21kworker/0:1+events21:07:270
1370088993025,3cyclictest1679652-21fschecks_time00:24:042
1370084993026,3cyclictest1510565-21kworker/u8:1+flush-8:022:29:011
1370084993025,3cyclictest1640371-21irqrtprio23:44:051
1370084993023,5cyclictest1456237-21kworker/u8:0+flush-8:021:19:131
1370084993022,6cyclictest1644566-21apt-get23:49:011
137008399307,18cyclictest1451058-21kworker/0:1+events22:40:030
1370083993027,2cyclictest1451044-21systemd-run20:30:010
137008399302,25cyclictest1451058-21kworker/0:1+events22:35:200
137008399302,24cyclictest1365038-21kworker/0:1+events19:42:110
137008399301,28cyclictest1451058-21kworker/0:1+events23:49:150
1370083993010,19cyclictest1365038-21kworker/0:1+events19:34:470
1370088992925,2cyclictest1500225-21cron21:20:002
1370084992925,3cyclictest1659273-21kworker/u8:4+events_unbound00:04:031
1370084992924,3cyclictest0-21swapper/123:59:061
1370084992923,5cyclictest1426175-21kworker/u8:1+flush-8:020:30:311
1370084992923,4cyclictest1668740-21unixbench_multi00:09:191
1370084992923,4cyclictest1623944-21unixbench_multi23:24:171
1370084992918,8cyclictest1525108-21latency_hist21:45:011
1370084992916,10cyclictest278-21systemd-journal23:54:271
137008399299,18cyclictest1451058-21kworker/0:1+events00:26:030
137008399298,19cyclictest1365038-21kworker/0:1+events20:09:110
137008399298,18cyclictest1451058-21kworker/0:1+events20:51:430
137008399295,22cyclictest1451058-21kworker/0:1+events23:29:070
137008399291,27cyclictest1451058-21kworker/0:1+events00:24:150
137008399291,23cyclictest1451058-21kworker/0:1+events21:50:550
137008399291,23cyclictest1431219-21kworker/0:0+events20:30:040
1370083992910,18cyclictest1451058-21kworker/0:1+events22:26:510
1370084992824,3cyclictest1451038-21kworker/u8:2+flush-8:020:59:151
1370084992823,3cyclictest1639207-21latency_hist23:40:011
1370084992823,3cyclictest1639207-21latency_hist23:40:001
1370084992820,5cyclictest1470778-21kworker/u8:3+flush-8:021:59:121
137008399283,20cyclictest1451058-21kworker/0:1+events23:05:080
137008399282,25cyclictest1451058-21kworker/0:1+events22:59:070
137008399282,25cyclictest1365038-21kworker/0:1+events19:15:110
137008399281,26cyclictest1451058-21kworker/0:1+events23:19:470
137008399281,23cyclictest1365038-21kworker/0:1+events19:55:150
137008399281,22cyclictest1451058-21kworker/0:1+events23:00:070
1370088992723,2cyclictest1619810-21aten2_r5power_p23:24:002
1370088992723,2cyclictest1520163-21cron21:39:592
1370088992723,2cyclictest1486129-21aten2_r5power_e21:09:012
1370084992723,3cyclictest1470778-21kworker/u8:3+flush-8:020:50:311
1370084992721,4cyclictest1658999-21kworker/u8:1+events_unbound00:15:001
137008399276,20cyclictest1451058-21kworker/0:1+events00:39:270
137008399276,19cyclictest1451058-21kworker/0:1+events21:32:470
137008399276,19cyclictest1451058-21kworker/0:1+events21:32:470
1370083992724,2cyclictest1494623-21needreboot21:14:090
1370083992724,2cyclictest1494623-21needreboot21:14:090
137008399272,24cyclictest1451058-21kworker/0:1+events21:39:470
137008399272,21cyclictest1451058-21kworker/0:1+events23:20:310
137008399272,20cyclictest1451058-21kworker/0:1+events00:06:430
1370083992721,4cyclictest1460817-21unixbench_singl20:39:240
137008399271,25cyclictest1451058-21kworker/0:1+events22:09:190
137008399271,21cyclictest1451058-21kworker/0:1+events23:50:230
1370088992623,2cyclictest1525729-21aten2_r5power_e21:49:062
1370088992622,2cyclictest1615108-21fschecks_time23:19:032
1370088992622,2cyclictest1437108-21fschecks_time20:19:032
1370084992622,3cyclictest277023-21kworker/u8:0+events_unbound19:10:051
1370084992621,3cyclictest1486790-21irqrtprio21:09:061
1370084992619,5cyclictest1426175-21kworker/u8:1+flush-8:020:19:181
1370084992617,7cyclictest1693525-21latency_hist00:35:001
1370084992616,8cyclictest1583596-21ntpq22:44:101
137008399269,16cyclictest1451058-21kworker/0:1+events21:04:030
137008399267,18cyclictest1451058-21kworker/0:1+events23:59:430
137008399264,17cyclictest1451058-21kworker/0:1+events22:00:110
137008399262,23cyclictest1451058-21kworker/0:1+events22:15:470
1370083992621,4cyclictest1634186-21apt-key23:35:000
137008399261,24cyclictest1365038-21kworker/0:1+events19:50:590
13679712263,16sleep00-21swapper/019:09:040
1370088992522,2cyclictest1431935-21aten2_r5power_e20:14:012
1370088992521,2cyclictest1664773-21cpuspeed_turbos00:09:022
1370088992520,4cyclictest1460907-21idleruntime-cro20:40:012
1370088992516,7cyclictest278-21systemd-journal23:30:002
1370084992521,3cyclictest1658999-21kworker/u8:1+flush-8:000:39:201
1370084992521,3cyclictest1609958-21cut23:14:031
1370084992521,2cyclictest0-21swapper/120:44:131
1370084992521,2cyclictest0-21swapper/100:24:071
1370084992519,4cyclictest1510565-21kworker/u8:1+flush-8:022:24:041
1370084992519,3cyclictest0-21swapper/119:29:151
1370084992515,7cyclictest278-21systemd-journal21:15:001
1370084992515,7cyclictest278-21systemd-journal21:15:001
137008399253,20cyclictest1365038-21kworker/0:1+events19:20:590
1370083992520,3cyclictest1678610-21unixbench_singl00:19:180
1370083992519,4cyclictest1693373-21unixbench_multi00:34:180
137008399251,23cyclictest1451058-21kworker/0:1+events22:48:310
137008399250,24cyclictest1451058-21kworker/0:1+events23:42:590
1370088992421,2cyclictest1480596-21idleruntime-cro21:00:002
1370088992419,4cyclictest904-21mta-sts-daemon21:29:092
1370088992415,7cyclictest1594302-21cp22:55:012
1370088992413,9cyclictest1658990-40logrotate00:00:002
137008899241,21cyclictest1455843-21unixbench_multi20:34:182
1370084992420,3cyclictest277023-21kworker/u8:0+flush-8:019:59:201
1370084992420,3cyclictest1588589-21kworker/u8:3+events_unbound22:59:171
1370084992420,3cyclictest1470778-21kworker/u8:3+flush-8:021:30:371
1370084992420,3cyclictest1470778-21kworker/u8:3+flush-8:021:30:371
1370084992420,2cyclictest0-21swapper/120:24:131
137008399247,16cyclictest1451058-21kworker/0:1+events00:03:070
137008399242,21cyclictest1451058-21kworker/0:1+events23:36:510
137008399242,21cyclictest1451058-21kworker/0:1+events23:36:510
1370088992320,2cyclictest1639763-21aten2_r5power_v23:44:002
1370088992319,2cyclictest1684498-21fschecks_count00:29:012
1370088992319,2cyclictest1530912-21fschecks_time21:54:042
1370088992319,2cyclictest0-21swapper/223:55:002
1370088992317,4cyclictest1648895-21apt-get23:50:002
1370084992319,3cyclictest1604133-21kworker/u8:1+events_unbound23:09:071
1370084992318,3cyclictest0-21swapper/123:04:051
1370084992318,3cyclictest0-21swapper/119:34:181
1370084992316,5cyclictest1411930-21apt-get19:53:591
137008899222,14cyclictest1579877-21apt-get22:44:012
1370088992219,2cyclictest1570027-21aten2_r5power_p22:34:022
1370088992219,2cyclictest1570027-21aten2_r5power_p22:34:012
1370088992218,2cyclictest0-21swapper/222:04:102
1370088992210,4cyclictest0-21swapper/220:24:012
1370084992218,3cyclictest1446061-21kworker/u8:3+flush-8:020:25:301
1370088992118,2cyclictest1481376-21cpuspeed_turbos21:04:012
1370088992116,3cyclictest0-21swapper/223:00:002
137008499212,5cyclictest1456543-21apt-get20:39:061
1370084992115,4cyclictest1594303-21latency_hist22:55:011
1370084992115,4cyclictest0-21swapper/120:14:161
1370084992114,5cyclictest0-21swapper/122:19:141
137008899209,3cyclictest0-21swapper/222:19:032
137008899200,8cyclictest15478991kthreadcore22:09:242
137008899200,18cyclictest1631183-21cut23:34:072
1370084992016,2cyclictest0-21swapper/123:19:111
1370084992015,3cyclictest0-21swapper/120:04:111
13650322203,8sleep30-21swapper/319:05:013
137008899198,4cyclictest0-21swapper/221:23:582
137008899194,2cyclictest16784452sort00:19:182
1370088991915,3cyclictest20643-21cron23:40:002
1370088991915,3cyclictest20643-21cron23:40:002
1370088991915,3cyclictest1515238-21cron21:35:002
1370088991915,3cyclictest1515238-21cron21:35:002
1370088991915,2cyclictest1421073-21unixbench_singl19:59:222
1370088991915,2cyclictest1387537-21cpuspeed_turbos19:29:022
1370088991915,2cyclictest0-21swapper/220:05:002
1370088991915,2cyclictest0-21swapper/219:39:182
1370088991913,4cyclictest1382393-21apt-get19:24:002
137008899189,7cyclictest1403659-21cut19:44:072
137008899182,12cyclictest1608523-21ntpq23:09:132
1370088991815,2cyclictest1689437-21fschecks_count00:34:032
1370088991814,3cyclictest20643-21cron21:15:002
1370088991814,3cyclictest20643-21cron21:15:002
1370088991814,2cyclictest1575237-21fschecks_count22:39:032
1370088991814,2cyclictest1520861-21aten2_r5power_v21:44:002
1370088991813,3cyclictest1449431-21kthreadcore20:29:092
1370088991813,1cyclictest0-21swapper/200:35:052
1370088991810,6cyclictest1414032-21tr19:54:082
137008899172,11cyclictest1395887-21ntpq19:34:102
1370088991714,2cyclictest0-21swapper/222:30:002
1370088991713,2cyclictest1604178-21ls23:05:002
137008899171,12cyclictest1539127-21missed_timers21:59:092
137008899162,2cyclictest0-21swapper/220:46:522
137008899162,13cyclictest0-21swapper/220:08:482
137008899162,13cyclictest0-21swapper/219:11:102
137008899162,13cyclictest0-21swapper/200:12:372
137008899162,13cyclictest0-21swapper/200:01:142
1370088991613,2cyclictest0-21swapper/222:22:382
1370088991613,2cyclictest0-21swapper/220:50:222
1370088991613,2cyclictest0-21swapper/220:43:302
1370088991612,2cyclictest1375646-21kthreadcore19:15:022
137008899161,11cyclictest1584812-21apt-get22:48:582
137008899152,10cyclictest1406974-21apt-get19:49:002
1370088991511,3cyclictest0-21swapper/222:14:082
16940842132,7sleep30-21swapper/300:39:023
16390632132,6sleep30-21swapper/323:40:013
16295232132,7sleep30-21swapper/323:33:593
16084312132,7sleep30-21swapper/323:09:113
15891572132,6sleep30-21swapper/322:50:013
15690062134,5sleep30-21swapper/322:29:153
15638862132,7sleep30-21swapper/322:24:143
15458062133,7sleep30-21swapper/322:09:213
15163252132,7sleep30-21swapper/321:39:363
14860442132,7sleep30-21swapper/321:08:593
16916202122,6sleep30-21swapper/300:34:093
16842382122,6sleep30-21swapper/300:28:593
16812742123,6sleep30-21swapper/300:24:103
16764202122,6sleep30-21swapper/300:19:083
16732712122,6sleep30-21swapper/300:14:153
16592632122,6sleep30-21swapper/300:00:273
16500632122,6sleep30-21swapper/323:54:053
16485322122,6sleep30-21swapper/323:49:193
16394322122,6sleep30-21swapper/323:41:293
16139882122,5sleep30-21swapper/323:15:013
15887762122,6sleep30-21swapper/322:49:173
15743672122,6sleep30-21swapper/322:35:013
15702272122,6sleep30-21swapper/322:34:053
15702272122,6sleep30-21swapper/322:34:053
15594622122,6sleep30-21swapper/322:19:203
15497102122,6sleep30-21swapper/322:10:013
15257852123,6sleep30-21swapper/321:49:083
15057142122,6sleep30-21swapper/321:29:013
14953322122,6sleep30-21swapper/321:15:013
14753032122,6sleep30-21swapper/320:54:183
14577462122,6sleep30-21swapper/320:39:143
14421242122,6sleep30-21swapper/320:24:053
14311352122,5sleep30-21swapper/320:10:013
14270832122,6sleep30-21swapper/320:09:053
14135042122,6sleep30-21swapper/319:54:083
13916602122,6sleep30-21swapper/319:30:013
13873062122,6sleep30-21swapper/319:29:013
13813002122,6sleep30-21swapper/319:19:143
13720252123,5sleep30-21swapper/319:14:043
16546892111,6sleep30-21swapper/323:59:023
16343652112,6sleep30-21swapper/323:35:033
16244802113,5sleep30-21swapper/323:27:183
16194752112,5sleep30-21swapper/323:21:073
15996032112,6sleep30-21swapper/323:02:053
15945642112,5sleep30-21swapper/322:55:233
15795562112,6sleep30-21swapper/322:40:103
15401862112,6sleep30-21swapper/322:01:033
15357222112,5sleep30-21swapper/321:59:023
15205882112,6sleep30-21swapper/321:42:133
15111372112,5sleep30-21swapper/321:34:023
15111372112,5sleep30-21swapper/321:34:023
15019332111,6sleep30-21swapper/321:24:073
14809812112,6sleep30-21swapper/321:02:013
14759382113,5sleep30-21swapper/320:55:353
14612572112,5sleep30-21swapper/320:41:353
14512552112,6sleep30-21swapper/320:30:113
14465852112,5sleep30-21swapper/320:28:193
14364862112,6sleep30-21swapper/320:15:233
14214402112,5sleep30-21swapper/320:00:183
14166332112,6sleep30-21swapper/319:56:533
14072142112,5sleep30-21swapper/319:49:033
13967792112,5sleep30-21swapper/319:35:073
13822672112,6sleep30-21swapper/319:22:443
15302662102,5sleep30-21swapper/321:50:033
14906302102,5sleep30-21swapper/321:10:033
14906302102,5sleep30-21swapper/321:10:033
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional