You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-13 - 05:40

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: ondemand
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack5slot4s.osadl.org (updated Sat Jun 13, 2026 00:44:13)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
289965421910,5sleep30-21swapper/300:25:353
2582537998476,5cyclictest2734609-21kworker/u8:1+flush-8:022:00:461
28954292790,5sleep20-21swapper/200:20:372
2582537997870,6cyclictest2728969-21kworker/u8:3+events_unbound21:40:451
2582537997266,4cyclictest2558222-21kworker/u8:1+events_unbound20:00:001
2582537997264,6cyclictest2758950-21kworker/u8:3+flush-8:023:20:431
2582534997066,3cyclictest1641715-21kworker/u8:0+events_unbound19:25:430
2582534996965,3cyclictest2734609-21kworker/u8:1+events_unbound22:10:280
2582534996965,3cyclictest1641715-21kworker/u8:0+events_unbound19:50:450
2582534996864,3cyclictest2558222-21kworker/u8:1+events_unbound19:40:300
2582537996762,4cyclictest2734609-21kworker/u8:1+events_unbound22:35:401
2582542996654,9cyclictest2907231-21ntpq00:30:422
2582534996662,3cyclictest2758950-21kworker/u8:3+events_unbound23:45:290
2582534996662,3cyclictest1641715-21kworker/u8:0+events_unbound19:45:380
27404092650,2sleep3471rcuc/321:45:343
2582537996559,4cyclictest2769053-21kworker/u8:0+flush-8:022:25:461
2582537996559,4cyclictest1641715-21kworker/u8:0+events_unbound20:15:441
2582534996561,3cyclictest2683954-21kworker/u8:2+events_unbound21:25:300
2582537996457,5cyclictest2758950-21kworker/u8:3+events_unbound23:35:411
2582537996456,6cyclictest2803679-21/usr/sbin/munin22:50:391
2582537996359,3cyclictest2728969-21kworker/u8:3+events_unbound21:50:391
2582537996358,3cyclictest2558222-21kworker/u8:1+flush-8:020:10:451
2582537996357,4cyclictest2683954-21kworker/u8:2+flush-8:021:55:451
2582537996356,4cyclictest2758950-21kworker/u8:3+flush-8:023:45:371
2582537996353,8cyclictest2599619-21/usr/sbin/munin19:25:361
2582534996358,4cyclictest2694052-21kworker/u8:3+flush-8:021:05:440
2582537996258,3cyclictest2558222-21kworker/u8:1+events_unbound20:05:421
2582534996257,4cyclictest2728969-21kworker/u8:3+events_unbound21:50:470
2582537996055,4cyclictest2694052-21kworker/u8:3+events_unbound21:05:421
2582537996054,5cyclictest2683954-21kworker/u8:2+flush-8:021:25:411
2582537996054,4cyclictest2769053-21kworker/u8:0+events_unbound22:45:291
2582534996056,3cyclictest2683954-21kworker/u8:2+events_unbound21:55:480
2582542995847,9cyclictest1704890-21systemd-journal19:50:412
2582537995853,3cyclictest2734609-21kworker/u8:1+flush-8:023:30:461
2582537995752,4cyclictest2734609-21kworker/u8:1+events_unbound00:25:391
2582537995747,8cyclictest2674298-21/usr/sbin/munin20:40:381
2582534995752,4cyclictest2734609-21kworker/u8:1+events_unbound22:00:490
2582537995651,4cyclictest1641715-21kworker/u8:0+flush-8:020:35:351
2582537995651,3cyclictest2758950-21kworker/u8:3+events_unbound00:00:311
2582537995548,5cyclictest1641715-21kworker/u8:0+events_unbound19:35:441
2582537995547,6cyclictest2855956-21kworker/u8:2+events_unbound00:10:501
2582534995545,8cyclictest2907180-21ntpq00:30:400
2582537995448,4cyclictest1641715-21kworker/u8:0+flush-8:020:45:411
2582537995448,4cyclictest1641715-21kworker/u8:0+flush-8:019:45:401
2582537995447,5cyclictest2803089-21kworker/u8:2+events_unbound23:05:291
2582537995446,6cyclictest2614445-21/usr/sbin/munin19:40:451
2582534995450,3cyclictest1641715-21kworker/u8:0+events_unbound20:45:330
28023932530,3chrt0-21swapper/222:45:382
2582537995349,3cyclictest2855956-21kworker/u8:2+flush-8:023:55:421
2582537995349,3cyclictest2683954-21kworker/u8:2+flush-8:021:00:441
2582537995349,3cyclictest2683954-21kworker/u8:2+events_unbound21:10:411
2582537995348,3cyclictest1641715-21kworker/u8:0+events_unbound20:00:441
2582537995347,4cyclictest2855956-21kworker/u8:2+flush-8:000:05:511
2582537995346,6cyclictest2703950-21kworker/u8:0+flush-8:021:15:421
2582537995346,5cyclictest2769053-21kworker/u8:0+events_unbound22:15:401
2582534995349,3cyclictest2683954-21kworker/u8:2+events_unbound21:15:030
2582534995344,4cyclictest2769053-21kworker/u8:0+events_unbound22:15:320
2582534995342,8cyclictest2724290-21/usr/sbin/munin21:30:360
2582537995247,3cyclictest1641715-21kworker/u8:0+flush-8:019:50:361
2582537995245,4cyclictest2703950-21kworker/u8:0+flush-8:021:30:311
2582534995248,3cyclictest2558222-21kworker/u8:1+events_unbound19:20:350
2582537995144,6cyclictest2558222-21kworker/u8:1+flush-8:020:25:411
2582542995044,4cyclictest2758950-21kworker/u8:3+flush-8:023:10:452
2582537995046,3cyclictest2758950-21kworker/u8:3+events_unbound22:05:441
2582537995046,3cyclictest2728969-21kworker/u8:3+events_unbound21:35:421
2582537995045,3cyclictest2803089-21kworker/u8:2+flush-8:023:15:451
2582534995046,3cyclictest2558222-21kworker/u8:1+events_unbound20:05:490
2582537994944,4cyclictest2855956-21kworker/u8:2+flush-8:000:20:421
2582537994944,3cyclictest1641715-21kworker/u8:0+flush-8:020:55:351
2582537994943,5cyclictest2728969-21kworker/u8:3+events_unbound21:45:411
2582534994945,3cyclictest2848010-21kworker/u8:0+events_unbound00:13:440
2582534994945,3cyclictest2758950-21kworker/u8:3+events_unbound22:20:470
2582542994843,3cyclictest2558222-21kworker/u8:1+events_unbound20:30:472
2582537994843,3cyclictest2758950-21kworker/u8:3+events_unbound23:00:451
2582534994844,3cyclictest2758950-21kworker/u8:3+events_unbound23:00:320
2582534994840,6cyclictest2823461-21apt-get23:10:030
2582534994840,3cyclictest2683954-21kworker/u8:2+events_unbound21:01:390
2582534994835,9cyclictest2873435-21cp00:00:000
2582537994743,3cyclictest2769053-21kworker/u8:0+events_unbound22:20:331
2582537994743,3cyclictest2558222-21kworker/u8:1+flush-8:020:50:301
2582537994742,3cyclictest2758950-21kworker/u8:3+events_unbound22:55:301
2582537994742,3cyclictest2558222-21kworker/u8:1+events_unbound19:35:011
2582537994740,5cyclictest2863484-21/usr/sbin/munin23:50:371
2582534994742,4cyclictest1641715-21kworker/u8:0+events_unbound19:57:410
2582542994642,3cyclictest2855956-21kworker/u8:2+events_unbound00:00:492
2582537994641,4cyclictest2769053-21kworker/u8:0+events_unbound22:40:271
2582534994642,3cyclictest2803089-21kworker/u8:2+events_unbound23:05:480
2582534994642,3cyclictest2734609-21kworker/u8:1+events_unbound00:36:430
2582534994637,7cyclictest2792670-21ntpq22:35:380
2582534994635,8cyclictest2857605-21python323:40:450
2582537994539,4cyclictest2848010-21kworker/u8:0+events_unbound23:40:291
2582537994539,4cyclictest2558222-21kworker/u8:1+events_unbound20:20:301
2582537994537,6cyclictest2558222-21kworker/u8:1+flush-8:019:15:421
2582534994541,3cyclictest2848010-21kworker/u8:0+events_unbound00:08:120
2582534994540,4cyclictest1641715-21kworker/u8:0+events_unbound20:18:400
2582537994434,8cyclictest2767661-21ntpq22:10:371
2582534994440,3cyclictest2758950-21kworker/u8:3+events_unbound23:17:390
2582534994440,3cyclictest2758950-21kworker/u8:3+events_unbound22:05:480
2582534994440,3cyclictest2743491-21kworker/u8:0+events_unbound21:48:380
2582534994440,3cyclictest2734609-21kworker/u8:1+events_unbound23:25:130
2582534994440,3cyclictest1641715-21kworker/u8:0+events_unbound20:35:540
2582534994440,3cyclictest1641715-21kworker/u8:0+events_unbound19:31:420
2582534994438,4cyclictest2558222-21kworker/u8:1+events_unbound20:50:270
2582542994339,3cyclictest2888523-21kworker/u8:3+events_unbound00:15:332
2582537994335,6cyclictest1704890-21systemd-journal20:34:591
2582534994339,3cyclictest2855956-21kworker/u8:2+events_unbound23:53:240
2582534994339,3cyclictest2855956-21kworker/u8:2+events_unbound00:28:440
2582534994339,3cyclictest2758950-21kworker/u8:3+events_unbound23:35:480
2582534994339,3cyclictest2758950-21kworker/u8:3+events_unbound22:59:400
2582534994339,3cyclictest2734609-21kworker/u8:1+events_unbound22:44:400
2582534994339,3cyclictest2558222-21kworker/u8:1+events_unbound19:37:410
2582534994339,3cyclictest1641715-21kworker/u8:0+events_unbound20:31:420
2582542994238,3cyclictest2558222-21kworker/u8:1+events_unbound19:10:362
2582537994238,3cyclictest2758950-21kworker/u8:3+flush-8:023:25:421
2582537994238,3cyclictest2758950-21kworker/u8:3+events_unbound23:10:411
2582534994238,3cyclictest2803089-21kworker/u8:2+events_unbound22:53:400
2582534994238,3cyclictest2758950-21kworker/u8:3+events_unbound22:26:390
2582534994238,3cyclictest2703950-21kworker/u8:0+events_unbound21:12:420
2582534994238,3cyclictest2558222-21kworker/u8:1+events_unbound19:12:310
2582534994238,3cyclictest2235101-21kworker/u8:2+events_unbound19:18:380
2582534994137,3cyclictest2758950-21kworker/u8:3+events_unbound23:21:400
2582534994137,3cyclictest2728969-21kworker/u8:3+events_unbound21:44:400
2582534994137,3cyclictest2683954-21kworker/u8:2+events_unbound21:35:270
2582534994137,3cyclictest2558222-21kworker/u8:1+events_unbound20:23:300
2582534994137,3cyclictest2558222-21kworker/u8:1+events_unbound20:10:540
2582534994137,3cyclictest1641715-21kworker/u8:0+events_unbound20:26:420
2582534994133,3cyclictest1641715-21kworker/u8:0+events_unbound20:59:420
27888442400,2sleep375-21kworker/3:1-mm_percpu_wq22:35:223
2582534994036,3cyclictest2888523-21kworker/u8:3+events_unbound00:20:490
2582534994036,3cyclictest2855956-21kworker/u8:2+events_unbound00:02:400
2582534994036,3cyclictest2734609-21kworker/u8:1+events_unbound23:32:400
2582534994036,3cyclictest2558222-21kworker/u8:1+events_unbound20:41:410
2582537993935,3cyclictest2582690-21kworker/u8:3+events_unbound19:10:271
2582534993935,3cyclictest1641715-21kworker/u8:0+events_unbound20:04:080
2582542993830,6cyclictest1704890-21systemd-journal21:55:012
2582534993833,4cyclictest2758950-21kworker/u8:3+events_unbound22:32:380
2582534993831,3cyclictest2734609-21kworker/u8:1+events_unbound22:47:080
2582534993733,3cyclictest2703950-21kworker/u8:0+events_unbound21:22:010
2582534993732,4cyclictest2855956-21kworker/u8:2+events_unbound00:16:550
2582542993631,3cyclictest2558222-21kworker/u8:1+events_unbound19:55:012
2582542993626,7cyclictest2822530-21ntpq23:05:372
2582542993625,8cyclictest2873385-21awk23:59:592
2582537993630,4cyclictest2903410-21apt-get00:30:261
2582542993429,3cyclictest2758950-21kworker/u8:3+events_unbound23:45:162
2582542993428,4cyclictest2888523-21kworker/u8:3+events_unbound00:25:012
26438562330,2sleep3481ktimers/320:09:593
2582542993325,6cyclictest2649289-21cat20:15:012
2582537993323,7cyclictest2892357-21ntpq00:15:401
25777592333,9sleep10-21swapper/119:05:061
2582542993228,3cyclictest2694052-21kworker/u8:3+events_unbound21:07:172
2582537993227,3cyclictest2558222-21kworker/u8:1+flush-8:019:20:371
2582542993127,3cyclictest2558222-21kworker/u8:1+events_unbound20:20:242
2582542993122,7cyclictest2639410-21cpuspeed_turbos20:05:292
2582537993126,4cyclictest2769053-21kworker/u8:0+events_unbound22:30:291
2582537993124,5cyclictest2908346-21apt-get00:35:271
258254299306,12cyclictest39-21ksoftirqd/220:20:002
2582542993026,3cyclictest2604611-21aten2_r5power_c19:30:282
2582542993024,4cyclictest2833562-21apt23:20:012
2582542993021,7cyclictest1704890-21systemd-journal21:00:012
2582542993018,2cyclictest161rcu_preempt19:20:362
258254299297,9cyclictest0-21swapper/200:10:382
258254299297,8cyclictest0-21swapper/223:35:452
258254299296,5cyclictest39-21ksoftirqd/220:55:002
2582542992917,4cyclictest161rcu_preempt20:00:362
258253499298,4cyclictest2582601-21ls19:10:000
258254299288,6cyclictest39-21ksoftirqd/220:39:592
2582542992825,2cyclictest2743414-21unixbench_multi21:45:462
2582542992824,2cyclictest2847912-21unixbench_multi23:30:472
2582542992823,3cyclictest2708330-21unixbench_singl21:10:472
2582542992819,7cyclictest2724629-21fschecks_time21:30:282
2582542992724,2cyclictest2783666-21cron22:30:002
2582542992723,2cyclictest2558222-21kworker/u8:1+events_unbound19:35:072
2582542992722,4cyclictest0-21swapper/222:30:392
2582542992716,2cyclictest161rcu_preempt22:55:342
258254299266,10cyclictest39-21ksoftirqd/223:05:012
2582542992623,2cyclictest2718446-21unixbench_multi21:20:492
2582542992622,3cyclictest2803788-21aten2_r5power_p22:50:282
2582542992622,3cyclictest1641715-21kworker/u8:0+events_unbound20:40:122
258254299260,7cyclictest2614698-21cpuspeed_turbos19:40:302
258254299253,5cyclictest0-21swapper/200:38:472
258254299252,5cyclictest2684007-21wc20:50:002
2582542992520,4cyclictest2734609-21kworker/u8:1-ext4-rsv-conversion22:00:002
258254299250,15cyclictest2867314-21grep23:50:402
2582537992516,5cyclictest2703950-21kworker/u8:0+events_unbound21:20:471
27372262240,4sleep30-21swapper/321:40:383
258254299243,4cyclictest0-21swapper/223:40:362
258254299242,5cyclictest2771281-21tr22:15:352
258254299242,5cyclictest0-21swapper/221:15:412
258254299242,5cyclictest0-21swapper/219:15:322
258254299242,2cyclictest161rcu_preempt22:37:522
2582542992420,3cyclictest1641715-21kworker/u8:0+events_unbound20:28:472
2582542992420,2cyclictest2734344-21aten2_r5power_v21:40:282
2582542992418,4cyclictest2623801-21journal-offline19:46:322
258254299240,12cyclictest2797663-21ld22:40:392
258254299238,4cyclictest0-21swapper/222:04:532
258254299236,2cyclictest0-21swapper/223:30:012
258254299233,8cyclictest0-21swapper/221:35:462
258254299233,4cyclictest0-21swapper/222:05:332
258254299232,6cyclictest2600357-21irqrtprio19:25:332
258254299232,5cyclictest0-21swapper/222:20:322
258254299232,4cyclictest0-21swapper/222:10:362
2582542992317,4cyclictest0-21swapper/221:00:302
258254299223,6cyclictest39-21ksoftirqd/221:30:002
258254299222,5cyclictest0-21swapper/200:05:372
258254299222,4cyclictest0-21swapper/223:20:322
25815662193,9sleep30-21swapper/319:05:443
25782962193,10sleep20-21swapper/219:05:352
28808752170,2sleep3471rcuc/300:05:453
26494672152,8sleep30-21swapper/320:15:273
28632172132,7sleep30-21swapper/323:50:013
28486232132,7sleep30-21swapper/323:35:283
28434512132,6sleep30-21swapper/323:30:023
28276002132,7sleep30-21swapper/323:10:393
27676992132,7sleep30-21swapper/322:10:403
27037002132,7sleep30-21swapper/321:10:013
26341432132,6sleep30-21swapper/320:00:023
26092612133,5sleep30-21swapper/319:35:023
29031552122,5sleep30-21swapper/300:30:013
28872362122,6sleep30-21swapper/300:10:433
28775922122,6sleep30-21swapper/300:00:443
28685302122,5sleep30-21swapper/323:55:013
28639332122,6sleep30-21swapper/323:50:333
28387762122,6sleep30-21swapper/323:25:283
28183192122,5sleep30-21swapper/323:05:023
28173512122,6sleep30-21swapper/323:00:413
28130622122,6sleep30-21swapper/322:56:393
28037082122,6sleep30-21swapper/322:50:283
27953692122,6sleep30-21swapper/322:40:343
27790402122,6sleep30-21swapper/322:25:293
27731462122,6sleep30-21swapper/322:15:463
27496192122,6sleep30-21swapper/321:55:353
27438462122,5sleep30-21swapper/321:50:023
27248092122,6sleep30-21swapper/321:30:303
27188652122,5sleep30-21swapper/321:25:023
27178062122,6sleep30-21swapper/321:20:403
27128232122,6sleep30-21swapper/321:15:383
27050762122,5sleep30-21swapper/321:10:363
26895422122,6sleep30-21swapper/320:55:313
26878672123,6sleep30-21swapper/320:50:383
26792862122,6sleep30-21swapper/320:45:293
26752522122,6sleep30-21swapper/320:40:343
26693302122,6sleep30-21swapper/320:35:293
26597372122,6sleep30-21swapper/320:25:303
26582512122,6sleep30-21swapper/320:20:403
26438562122,5sleep30-21swapper/320:10:013
26292982122,6sleep30-21swapper/319:55:283
26153662122,6sleep30-21swapper/319:40:363
26054982122,6sleep30-21swapper/319:30:353
25864192123,6sleep30-21swapper/319:10:383
29128142112,6sleep30-21swapper/300:37:323
28934972112,5sleep30-21swapper/300:20:023
28885792112,5sleep30-21swapper/300:15:283
28536502112,5sleep30-21swapper/323:40:293
28324962112,5sleep30-21swapper/323:15:373
27991252112,5sleep30-21swapper/322:45:323
27845342112,5sleep30-21swapper/322:30:353
27744812113,5sleep30-21swapper/322:20:313
27589392112,5sleep30-21swapper/322:05:033
27577932112,5sleep30-21swapper/322:00:403
27292372112,5sleep30-21swapper/321:35:293
26944862112,5sleep30-21swapper/321:00:343
26681682112,6sleep30-21swapper/320:30:403
26243032112,5sleep30-21swapper/319:50:023
26194222112,5sleep30-21swapper/319:45:253
25995892112,5sleep30-21swapper/319:25:033
25946582112,5sleep30-21swapper/319:20:293
25878902112,5sleep30-21swapper/319:15:023
28376852102,5sleep30-21swapper/323:20:433
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional