You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-08-27 - 09:31

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: ondemand
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack5slot4s.osadl.org (updated Wed Aug 27, 2025 00:44:12)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
10453632850,2sleep3471rcuc/323:35:333
9700882800,3chrt75-21kworker/3:1-events22:20:343
778688996151,7cyclictest1024398-21latency_hist23:15:010
778688995645,8cyclictest1064007-21apt-get23:55:000
778692995549,4cyclictest879613-21latency_hist20:50:011
778688995242,7cyclictest989325-21latency_hist22:40:000
778688994844,3cyclictest805011-21kworker/u8:2+events_unbound19:50:420
778692994539,4cyclictest795263-21kworker/u8:1+events_unbound19:25:331
778692994535,8cyclictest939520-21/usr/sbin/munin21:50:401
8366532440,3sleep1281rcuc/120:05:341
778692994435,6cyclictest276-21systemd-journal19:35:481
778688994438,4cyclictest999745-21apt-get22:50:240
778688994434,7cyclictest835609-21diskstats20:05:300
778694994338,3cyclictest955051-21kworker/u8:1+flush-8:000:10:302
778692994235,5cyclictest825491-21apt-get19:55:251
778688994232,7cyclictest276-21systemd-journal20:34:480
7934652410,5sleep30-21swapper/319:20:353
778694994137,2cyclictest914322-21kworker/u8:2+flush-8:021:45:282
778694994135,4cyclictest840010-21kworker/u8:3+flush-8:021:00:452
778692994135,4cyclictest879888-21apt-get20:50:241
778688994134,5cyclictest978202-21ntpq22:25:370
77868899412,29cyclictest829253-21ntpq19:55:390
778692994033,5cyclictest1103697-40mandb00:32:201
778692994030,7cyclictest276-21systemd-journal22:41:481
8252052390,4sleep2161rcu_preempt19:55:002
778692993933,4cyclictest1053962-21kworker/u8:2+flush-8:023:50:351
778692993932,5cyclictest865054-21apt-get20:35:291
778692993927,8cyclictest276-21systemd-journal21:20:031
778688993930,6cyclictest984083-21latency_hist22:35:000
778688993929,8cyclictest830672-21diskstats20:00:280
778692993831,5cyclictest1064515-21apt-get23:55:241
77868899385,11cyclictest1018606-21ntpq23:05:420
778688993831,5cyclictest276-21systemd-journal20:17:360
77868899382,13cyclictest791045-21hddtemp_smartct19:20:300
778692993730,5cyclictest4116332-21kworker/u8:0+flush-8:020:10:311
778688993728,7cyclictest1088154-21ntpq00:15:360
778694993630,4cyclictest1038651-21kworker/u8:0+events_unbound23:35:442
778692993627,7cyclictest958437-21ntpq22:05:391
778692993626,7cyclictest276-21systemd-journal00:06:481
778688993631,4cyclictest854891-21kworker/u8:1+events_unbound21:25:270
778688993631,4cyclictest854891-21kworker/u8:1+events_unbound21:25:270
77868899362,26cyclictest0-21swapper/023:02:170
778688993616,9cyclictest585-21runrttasks23:35:410
778694993526,7cyclictest815227-21gpgv19:45:012
778694993524,8cyclictest969217-21awk22:20:002
778692993528,5cyclictest276-21systemd-journal20:22:351
778692993528,5cyclictest1044222-21cat23:35:001
778688993518,4cyclictest993445-21ntpq22:40:440
778694993425,7cyclictest1044217-21turbostat.cron23:35:002
778688993429,4cyclictest805011-21kworker/u8:2+flush-8:019:36:530
778688993425,7cyclictest949253-21cp22:00:000
77868899342,20cyclictest890005-21diskstats21:00:300
77868899340,23cyclictest783884-21users19:15:010
9945432330,2sleep3471rcuc/322:45:013
8653772330,3chrt451irq_work/320:35:333
778694993329,2cyclictest869218-21timerwakeupswit20:35:502
77869499332,7cyclictest904594-21apt-get21:15:282
778692993328,3cyclictest980143-21irqrtprio22:30:351
778692993327,4cyclictest276-21systemd-journal23:27:571
778692993323,8cyclictest276-21systemd-journal21:00:001
77868899332,6cyclictest0-21swapper/022:59:140
778688993324,6cyclictest844973-21latency_hist20:15:000
778688993323,8cyclictest1068233-21ntpq23:55:370
778694993227,4cyclictest955051-21kworker/u8:1+flush-8:023:40:482
778692993227,3cyclictest810461-21apt-get19:40:241
778692993227,3cyclictest1005378-21irqrtprio22:55:311
778692993227,3cyclictest1004377-21cron22:55:001
778692993223,7cyclictest276-21systemd-journal21:32:461
778692993223,7cyclictest1023465-21ntpq23:10:411
778692993221,8cyclictest909238-21latency_hist21:20:001
778692993221,8cyclictest276-21systemd-journal23:00:481
778688993228,3cyclictest1029328-21cron23:20:000
778688993222,7cyclictest939266-21latency_hist21:50:010
77868899321,9cyclictest799614-21python319:25:430
77868899320,17cyclictest880415-21ps20:50:310
778694993127,2cyclictest835472-21aten2_r5power_v20:05:282
778694993122,7cyclictest276-21systemd-journal21:55:482
778692993126,3cyclictest1025406-21irqrtprio23:15:351
778692993122,5cyclictest0-21swapper/100:15:251
77868899314,17cyclictest1034816-21df23:25:310
778688993125,4cyclictest924361-21kworker/u8:0+flush-8:021:43:520
778688993122,6cyclictest1074167-21latency_hist00:05:000
778688993120,8cyclictest969203-21latency_hist22:20:010
778688993112,4cyclictest1102978-21ntpq00:30:380
778688993112,18cyclictest810204-21kworker/0:1+events20:23:280
77868899310,7cyclictest810462-21dpkg19:40:240
77868899310,6cyclictest900167-21irqcore21:10:320
77868899310,13cyclictest857223-21cut20:25:330
77868899310,11cyclictest1094437-21cpu00:25:280
778694993026,3cyclictest4127157-21kworker/u8:3+events_unbound19:15:442
778694993025,4cyclictest978964-21kworker/u8:3+flush-8:022:32:492
778694993025,3cyclictest1059405-21aten2_r5power_e23:50:272
778692993025,3cyclictest935306-21irqrtprio21:45:321
778692993024,5cyclictest1069956-21kworker/u8:0+flush-8:000:12:191
778692993022,6cyclictest4127157-21kworker/u8:3+flush-8:019:20:401
778692993022,6cyclictest1054417-21apt-get23:45:291
77868899306,11cyclictest1040046-21irqcore23:30:310
778688993025,3cyclictest1094068-21systemd-run00:25:020
77868899302,19cyclictest0-21swapper/021:50:380
778688993021,6cyclictest869723-21latency_hist20:40:010
778688993013,9cyclictest0-21swapper/021:20:010
778688993011,10cyclictest803678-21cut19:30:350
77868899300,3cyclictest961040-21kthreadcore22:10:320
77868899300,16cyclictest929361-21systemd-run21:40:010
77868899300,12cyclictest1104878-21irqcore00:35:310
778694992925,2cyclictest1099588-21fschecks_time00:30:312
778694992924,3cyclictest955051-21kworker/u8:1+flush-8:023:00:332
778692992925,2cyclictest1091543-21kthreadcore00:20:331
778692992924,3cyclictest825236-21systemd-run19:55:011
778692992921,6cyclictest929602-21/usr/sbin/munin21:40:421
778692992919,7cyclictest1069956-21kworker/u8:0+flush-8:000:00:431
77868899295,10cyclictest953462-21ntpq22:00:400
778688992925,3cyclictest955051-21kworker/u8:1+events_unbound23:40:340
778688992923,4cyclictest919757-21/usr/sbin/munin21:30:250
778688992918,2cyclictest161rcu_preempt23:20:330
778688992911,7cyclictest894769-21cut21:05:270
77868899290,12cyclictest873801-21ntpq20:40:380
778694992823,3cyclictest959849-21fschecks_count22:10:282
778694992820,6cyclictest1094033-21cron00:25:002
778692992824,3cyclictest291ktimers/120:30:371
778692992821,5cyclictest988414-21ntpq22:35:381
778692992818,7cyclictest782602-21ld19:10:361
77868899285,2cyclictest1083314-21ntpq00:10:390
77868899285,10cyclictest1078360-21ntpq00:05:420
77868899284,13cyclictest911751-21cut21:20:330
778688992825,2cyclictest995502-21irqrtprio22:45:320
77868899282,15cyclictest958658-21python322:05:440
778688992814,5cyclictest0-21swapper/023:45:420
77868899281,16cyclictest875334-21hddtemp_smartct20:45:340
778694992723,3cyclictest955051-21kworker/u8:1+flush-8:022:40:452
778694992723,2cyclictest1054528-21aten2_r5power_v23:45:322
77869299273,22cyclictest891051-21kthreadcore21:00:341
778692992722,3cyclictest783739-21munin-node22:10:311
778692992722,3cyclictest0-21swapper/123:40:011
778692992722,3cyclictest0-21swapper/122:45:291
778692992721,4cyclictest903930-21unixbench_singl21:10:451
778692992721,3cyclictest0-21swapper/100:35:311
77869299270,24cyclictest830397-21acpi20:00:241
77869299270,24cyclictest1014750-21gpgv23:05:011
77868899274,11cyclictest818876-21cut19:45:360
778688992717,2cyclictest161rcu_preempt20:55:330
77868899270,11cyclictest782230-21cut19:10:330
778694992621,4cyclictest1069956-21kworker/u8:0+flush-8:000:00:382
778694992616,7cyclictest1109110-21cat00:40:012
77869499260,24cyclictest954220-21taskset22:03:312
778692992620,4cyclictest1033465-21ntpq23:20:451
77868899268,2cyclictest161rcu_preempt22:20:300
778694992521,3cyclictest0-21swapper/222:57:352
778694992521,3cyclictest0-21swapper/220:20:282
778694992521,2cyclictest0-21swapper/219:50:002
778694992520,4cyclictest1019471-21idleruntime-cro23:10:002
77869499251,11cyclictest9281792cut21:35:382
77869299252,22cyclictest0-21swapper/123:43:041
778692992521,2cyclictest291ktimers/121:58:301
778692992521,2cyclictest0-21swapper/121:10:001
778692992520,4cyclictest805011-21kworker/u8:2+flush-8:019:32:361
778694992421,2cyclictest0-21swapper/223:56:292
778694992421,2cyclictest0-21swapper/223:14:232
778694992421,2cyclictest0-21swapper/222:51:022
778694992421,2cyclictest0-21swapper/222:28:232
778694992421,2cyclictest0-21swapper/222:08:042
778694992421,2cyclictest0-21swapper/221:51:312
778694992421,2cyclictest0-21swapper/221:41:402
778694992421,2cyclictest0-21swapper/220:55:412
778694992421,2cyclictest0-21swapper/220:52:252
778694992421,2cyclictest0-21swapper/220:49:072
778694992421,2cyclictest0-21swapper/220:12:592
778694992421,2cyclictest0-21swapper/220:03:082
778694992421,2cyclictest0-21swapper/219:56:342
778694992421,2cyclictest0-21swapper/200:26:462
778694992421,2cyclictest0-21swapper/200:09:442
778694992420,3cyclictest995101-21fschecks_time22:45:292
778694992420,2cyclictest805564-21aten2_r5power_e19:35:272
778694992420,2cyclictest1085352-21kthreadcore00:15:322
778694992419,4cyclictest800658-21aten2_r5power_v19:30:272
77869299242,20cyclictest0-21swapper/122:00:351
77869299242,20cyclictest0-21swapper/100:25:321
778692992421,2cyclictest0-21swapper/122:28:541
778692992421,2cyclictest0-21swapper/122:21:371
778692992421,2cyclictest0-21swapper/122:18:201
778692992421,2cyclictest0-21swapper/121:29:031
778692992421,2cyclictest0-21swapper/121:29:031
778692992421,2cyclictest0-21swapper/120:43:041
778692992421,2cyclictest0-21swapper/120:16:401
778692992419,3cyclictest819864-21unixbench_multi19:45:461
778692992418,4cyclictest928919-21unixbench_multi21:35:471
77869499232,3cyclictest0-21swapper/223:25:342
77869499232,20cyclictest0-21swapper/221:20:492
77869499232,20cyclictest0-21swapper/221:10:102
77869499232,20cyclictest0-21swapper/219:25:182
778694992320,2cyclictest0-21swapper/223:20:132
778694992320,2cyclictest0-21swapper/223:16:562
778694992320,2cyclictest0-21swapper/222:37:172
778694992320,2cyclictest0-21swapper/221:25:072
778694992320,2cyclictest0-21swapper/221:25:072
778694992320,2cyclictest0-21swapper/221:05:182
778694992320,2cyclictest0-21swapper/220:32:192
778694992320,2cyclictest0-21swapper/220:26:002
778694992320,2cyclictest0-21swapper/219:20:112
778694992320,2cyclictest0-21swapper/219:10:202
778692992320,2cyclictest0-21swapper/120:26:171
778692992320,2cyclictest0-21swapper/119:17:111
7782192233,14sleep00-21swapper/019:06:480
8397802222,16sleep30-21swapper/320:07:303
77869499222,2cyclictest0-21swapper/222:21:192
77869499222,2cyclictest0-21swapper/220:40:092
778694992219,2cyclictest0-21swapper/221:31:122
778694992219,2cyclictest0-21swapper/220:15:462
7738562214,11sleep10-21swapper/119:05:201
7784692206,10sleep20-21swapper/219:09:592
10301932200,2chrt75-21kworker/3:1-events23:20:363
9342782192,14sleep30-21swapper/321:44:523
8646762192,14sleep30-21swapper/320:33:583
8596162192,9sleep30-21swapper/320:27:283
7784482193,9sleep30-21swapper/319:09:453
9592452182,13sleep30-21swapper/322:09:243
7856902152,8sleep30-21swapper/319:15:263
9491202133,7sleep30-21swapper/322:00:003
8762462133,6sleep30-21swapper/320:45:383
11047832132,7sleep30-21swapper/300:35:323
9968092122,6sleep30-21swapper/322:45:343
9642782122,5sleep30-21swapper/322:15:013
9631952122,6sleep30-21swapper/322:10:363
9388592122,6sleep30-21swapper/321:45:443
9203692122,6sleep30-21swapper/321:30:313
9146762122,6sleep30-21swapper/321:25:263
9146762122,6sleep30-21swapper/321:25:263
9097362122,6sleep30-21swapper/321:20:243
8951062122,6sleep30-21swapper/321:05:323
8842952122,6sleep30-21swapper/320:51:593
8452602122,6sleep30-21swapper/320:15:283
8297692122,6sleep30-21swapper/319:55:463
8059662122,6sleep30-21swapper/319:35:323
10794662122,6sleep30-21swapper/300:10:243
10692292122,5sleep30-21swapper/300:00:013
10595962122,6sleep30-21swapper/323:50:303
10431492122,6sleep30-21swapper/323:30:363
10246552122,6sleep30-21swapper/323:15:273
10194882122,5sleep30-21swapper/323:10:023
10142812121,7sleep30-21swapper/323:05:003
10043662122,6sleep30-21swapper/322:55:013
9884212113,5sleep30-21swapper/322:35:393
9797962112,6sleep30-21swapper/322:30:343
9788042112,6sleep30-21swapper/322:25:523
9536142112,5sleep30-21swapper/322:00:453
9392542112,4sleep30-21swapper/321:50:023
9249092112,5sleep30-21swapper/321:35:343
9050792112,6sleep30-21swapper/321:15:353
9003542112,5sleep30-21swapper/321:10:333
8897012112,5sleep30-21swapper/321:00:023
8858622112,5sleep30-21swapper/320:55:333
8738502112,5sleep30-21swapper/320:40:403
8538462112,5sleep30-21swapper/320:20:383
8402482112,5sleep30-21swapper/320:10:273
8303952112,5sleep30-21swapper/320:00:243
8167622110,7sleep30-21swapper/319:45:353
8101982112,5sleep30-21swapper/319:40:013
7953542112,5sleep30-21swapper/319:25:013
7787962112,5sleep30-21swapper/319:10:203
10991792111,5sleep30-21swapper/300:30:033
10942962112,5sleep30-21swapper/300:25:253
10929112112,5sleep30-21swapper/300:20:353
10844312112,6sleep30-21swapper/300:15:263
10744482112,5sleep30-21swapper/300:05:273
10683112112,6sleep30-21swapper/323:55:403
10532592112,5sleep30-21swapper/323:40:443
10345192112,5sleep30-21swapper/323:25:033
10147542112,5sleep30-21swapper/323:05:043
10037262112,5sleep30-21swapper/322:50:443
8242072102,4sleep30-21swapper/319:50:373
8004902102,5sleep30-21swapper/319:30:023
10549162102,5sleep30-21swapper/323:45:373
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional