You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-02 - 09:19

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: ondemand
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack5slot4s.osadl.org (updated Thu Apr 02, 2026 00:44:17)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
86838821150,3sleep1281rcuc/100:36:331
8072892710,6sleep30-21swapper/323:36:273
7536942680,3sleep0201rcuc/022:41:300
540941996864,3cyclictest701025-21kworker/u8:2+events_unbound21:56:401
540941996460,3cyclictest786651-21kworker/u8:3+flush-8:000:11:401
540941996359,3cyclictest786651-21kworker/u8:3+flush-8:023:56:411
540941996258,3cyclictest671823-21kworker/u8:0+events_unbound22:26:421
540941996157,3cyclictest766468-21kworker/u8:1+events_unbound23:16:401
540939996045,11cyclictest280-21systemd-journal21:01:260
6827302590,2sleep375-21kworker/3:1-events21:31:273
6827302590,2sleep375-21kworker/3:1-events21:31:273
540941995854,3cyclictest498972-21kworker/u8:0+events_unbound20:06:391
540939995849,7cyclictest572866-21latency_hist19:40:000
540941995752,3cyclictest557602-21kworker/u8:3+events_unbound21:15:011
540941995450,3cyclictest746689-21kworker/u8:2+events_unbound23:10:001
7735732530,3chrt371rcuc/223:01:312
540941995349,3cyclictest766468-21kworker/u8:1+events_unbound00:00:321
540941995349,3cyclictest766468-21kworker/u8:1+events_unbound00:00:321
540939995344,6cyclictest851261-21latency_hist00:20:010
540939995330,20cyclictest860272-21ntpq00:26:330
540939995240,8cyclictest567723-21latency_hist19:35:000
540939995240,8cyclictest567723-21latency_hist19:35:000
8634842510,4sleep3451irq_work/300:31:293
540944995141,7cyclictest280-21systemd-journal23:25:002
540941995147,3cyclictest786651-21kworker/u8:3+events_unbound23:46:261
540941995147,3cyclictest671823-21kworker/u8:0+flush-8:022:46:371
540941995046,3cyclictest622834-21kworker/u8:2+events_unbound20:35:381
540941995045,4cyclictest540808-21kworker/u8:2+events_unbound19:16:301
540941995045,3cyclictest746689-21kworker/u8:2+events_unbound22:40:151
540939995041,7cyclictest841350-21latency_hist00:10:000
540944994944,3cyclictest671823-21kworker/u8:0+flush-8:021:41:382
540941994945,3cyclictest509193-21kworker/u8:1+events_unbound20:11:451
540939994931,15cyclictest280-21systemd-journal21:11:290
540941994845,2cyclictest509193-21kworker/u8:1+events_unbound19:26:391
540941994845,2cyclictest509193-21kworker/u8:1+events_unbound19:26:391
540941994844,3cyclictest671823-21kworker/u8:0+events_unbound22:21:411
540941994840,6cyclictest816501-21latency_hist23:45:011
540939994837,8cyclictest556742-21ntpq19:21:350
540939994837,8cyclictest556742-21ntpq19:21:350
8624752470,4sleep20-21swapper/200:31:282
540941994743,3cyclictest498972-21kworker/u8:0+events_unbound20:45:151
540941994742,4cyclictest498972-21kworker/u8:0+events_unbound20:43:381
540939994727,17cyclictest637441-21latency_hist20:45:010
540944994638,6cyclictest280-21systemd-journal19:30:002
540944994638,6cyclictest280-21systemd-journal19:30:002
540941994642,3cyclictest701025-21kworker/u8:2+events_unbound22:14:381
540941994641,4cyclictest661949-21kworker/u8:2+events_unbound21:36:361
540941994635,9cyclictest280-21systemd-journal21:20:001
540939994637,6cyclictest672106-21latency_hist21:20:010
540939994624,3cyclictest866088-21sh00:35:000
540944994534,8cyclictest280-21systemd-journal19:20:002
540941994541,3cyclictest766468-21kworker/u8:1+events_unbound22:53:381
540941994541,3cyclictest701025-21kworker/u8:2+events_unbound22:01:391
540941994541,3cyclictest671823-21kworker/u8:0+events_unbound22:31:251
540941994541,3cyclictest652467-21kworker/u8:1+events_unbound21:45:141
540941994541,3cyclictest509193-21kworker/u8:1+events_unbound19:41:391
540941994541,3cyclictest498972-21kworker/u8:0+events_unbound21:06:241
540941994540,4cyclictest652467-21kworker/u8:1+events_unbound21:26:401
540941994534,8cyclictest280-21systemd-journal22:57:371
540939994533,9cyclictest280-21systemd-journal19:50:010
540939994526,6cyclictest866798-21fschecks_time00:36:280
540944994432,8cyclictest581930-21ntpq19:46:352
540941994440,3cyclictest746689-21kworker/u8:2+events_unbound23:11:341
540941994440,3cyclictest671823-21kworker/u8:0+events_unbound21:40:151
540941994440,3cyclictest557602-21kworker/u8:3+events_unbound20:21:191
540941994440,3cyclictest498972-21kworker/u8:0+events_unbound21:02:161
540941994435,7cyclictest280-21systemd-journal21:35:001
540941994435,7cyclictest280-21systemd-journal21:35:001
540941994430,7cyclictest612457-21awk20:20:001
540939994425,17cyclictest736705-21systemd-run22:25:010
540939994418,3cyclictest0-21swapper/021:25:000
540941994339,3cyclictest846041-21kworker/u8:0+events_unbound00:31:391
540941994339,3cyclictest766468-21kworker/u8:1+events_unbound23:01:591
540941994339,3cyclictest498972-21kworker/u8:0+events_unbound20:55:151
540939994326,5cyclictest787465-21grep23:16:250
540939994326,15cyclictest545794-21cron19:15:000
540941994238,3cyclictest846041-21kworker/u8:0+flush-8:000:16:351
540941994233,6cyclictest751973-21latency_hist22:40:011
540939994224,5cyclictest280-21systemd-journal19:56:150
540939994221,6cyclictest726510-21latency_hist22:15:010
540941994137,3cyclictest786651-21kworker/u8:3+events_unbound23:20:141
540941994137,3cyclictest786651-21kworker/u8:3+events_unbound00:06:271
540941994137,3cyclictest498972-21kworker/u8:0+events_unbound20:01:361
540941994136,4cyclictest652467-21kworker/u8:1+events_unbound21:51:591
540941994037,2cyclictest671823-21kworker/u8:0+events_unbound21:21:391
540941994036,3cyclictest766468-21kworker/u8:1+events_unbound23:31:231
540941994036,3cyclictest509193-21kworker/u8:1+events_unbound19:53:361
540941994036,3cyclictest509193-21kworker/u8:1+events_unbound19:35:121
540941994036,3cyclictest498972-21kworker/u8:0+events_unbound20:26:261
540939994031,6cyclictest716641-21latency_hist22:05:010
540939994018,6cyclictest280-21systemd-journal00:05:000
540939994018,6cyclictest280-21systemd-journal00:05:000
540941993935,3cyclictest766468-21kworker/u8:1+events_unbound23:51:411
540941993935,3cyclictest766468-21kworker/u8:1+events_unbound23:51:411
540941993935,3cyclictest701025-21kworker/u8:2+events_unbound22:09:371
540941993935,3cyclictest509193-21kworker/u8:1+events_unbound19:46:341
540941993935,3cyclictest509193-21kworker/u8:1+events_unbound19:21:111
540941993935,3cyclictest509193-21kworker/u8:1+events_unbound19:21:111
540941993934,4cyclictest786651-21kworker/u8:3+events_unbound00:25:021
540939993931,6cyclictest607818-21/usr/sbin/munin20:16:320
540941993834,3cyclictest557602-21kworker/u8:3+events_unbound20:50:391
540941993832,4cyclictest567729-21apt-get19:35:001
540941993832,4cyclictest567729-21apt-get19:35:001
540939993831,5cyclictest280-21systemd-journal20:11:170
540939993830,6cyclictest815637-21ntpq23:41:340
540939993829,6cyclictest647886-21diskstats20:56:250
540939993828,7cyclictest731446-21apt-get22:20:000
540944993729,6cyclictest280-21systemd-journal23:15:002
540941993733,3cyclictest786651-21kworker/u8:3+events_unbound23:35:381
540941993733,3cyclictest498972-21kworker/u8:0+events_unbound20:31:271
5361102374,11sleep10-21swapper/119:05:101
540944993630,4cyclictest671823-21kworker/u8:0+flush-8:022:11:332
540941993632,3cyclictest786651-21kworker/u8:3+events_unbound00:22:161
540941993531,3cyclictest786651-21kworker/u8:3+events_unbound23:27:381
540941993529,4cyclictest730654-21kworker/u8:3+events_unbound22:17:371
540939993528,5cyclictest548062-21apt-get19:16:230
540941993430,3cyclictest498972-21kworker/u8:0+events_unbound19:55:031
540939993426,6cyclictest280-21systemd-journal23:36:580
540939993423,8cyclictest280-21systemd-journal23:55:000
540939993423,8cyclictest280-21systemd-journal23:55:000
7327142330,5sleep3451irq_work/322:21:313
540941993329,3cyclictest498972-21kworker/u8:0+events_unbound19:11:331
540939993324,7cyclictest772395-21diskstats23:01:260
540939993324,6cyclictest701795-21latency_hist21:50:010
540939993310,15cyclictest530790-21kworker/0:0+events19:26:220
540939993310,15cyclictest530790-21kworker/0:0+events19:26:220
540944993222,8cyclictest280-21systemd-journal20:27:372
540944993222,7cyclictest612759-21cat20:20:012
540939993220,9cyclictest280-21systemd-journal00:25:010
54093999321,29cyclictest741648-21sshd22:27:370
540944993125,4cyclictest736360-21unixbench_singl22:21:442
540939993121,7cyclictest786625-21apt-get23:15:000
540944993022,6cyclictest280-21systemd-journal00:06:372
540944993019,8cyclictest280-21systemd-journal22:10:002
540939993026,2cyclictest721571-21idleruntime-cro22:10:000
54093999302,27cyclictest592687-21kworker/0:0+events20:21:330
54093999302,26cyclictest592687-21kworker/0:0+events23:30:000
540939993021,7cyclictest831430-21cp00:00:000
540939992919,7cyclictest706780-21latency_hist21:55:010
540944992824,2cyclictest557271-21unixbench_singl19:21:422
540944992824,2cyclictest557271-21unixbench_singl19:21:422
540944992823,4cyclictest498972-21kworker/u8:0+flush-8:020:36:392
540944992821,5cyclictest280-21systemd-journal22:30:012
540944992821,5cyclictest280-21systemd-journal20:45:002
540944992820,5cyclictest280-21systemd-journal19:15:002
540944992819,7cyclictest280-21systemd-journal21:51:372
540939992822,4cyclictest711402-21unixbench_singl21:56:420
540939992822,4cyclictest642093-21unixbench_singl20:46:420
54093999281,26cyclictest592687-21kworker/0:0+events22:36:340
540939992810,16cyclictest562574-21kworker/0:2+events19:41:330
540944992723,3cyclictest661885-21unixbench_singl21:06:422
540944992723,2cyclictest856665-21aten2_r5power_v00:26:232
54093999275,20cyclictest592687-21kworker/0:0+events21:31:340
54093999275,20cyclictest592687-21kworker/0:0+events21:31:340
540939992723,3cyclictest742917-21sed22:31:290
540939992710,15cyclictest592687-21kworker/0:0+events23:21:260
540939992710,15cyclictest592687-21kworker/0:0+events20:36:340
540944992623,2cyclictest797230-21aten2_r5power_v23:26:242
540944992620,4cyclictest597956-21apt-get20:06:222
540944992612,9cyclictest866447-21apt-get00:36:252
54093999269,15cyclictest592687-21kworker/0:0+events20:51:250
540939992622,3cyclictest766468-21kworker/u8:1+flush-8:023:06:080
540939992622,2cyclictest0-21swapper/023:00:000
540939992610,15cyclictest592687-21kworker/0:0+events21:26:380
540944992521,2cyclictest677515-21aten2_r5power_p21:26:242
540944992520,4cyclictest746689-21kworker/u8:2+flush-8:023:36:372
540944992520,4cyclictest557602-21kworker/u8:3+flush-8:021:16:372
54093999258,15cyclictest592687-21kworker/0:0+events21:41:380
540939992520,3cyclictest846317-21systemd-run00:15:010
540939992517,6cyclictest280-21systemd-journal22:47:370
54093999250,23cyclictest687552-21tr21:36:260
540944992420,2cyclictest647953-21fschecks_time20:56:252
540944992419,3cyclictest613270-21cpuspeed_turbos20:21:252
540944992413,3cyclictest0-21swapper/220:35:002
54093999244,17cyclictest592687-21kworker/0:0+events23:31:220
540939992419,3cyclictest621940-21mailstats20:26:400
5405432243,4sleep00-21swapper/019:07:300
540944992319,3cyclictest766468-21kworker/u8:1+flush-8:000:05:002
540944992319,3cyclictest766468-21kworker/u8:1+flush-8:000:04:592
540944992319,2cyclictest0-21swapper/220:05:002
540944992316,5cyclictest280-21systemd-journal19:45:012
54093999238,13cyclictest592687-21kworker/0:0+events20:01:280
54093999234,17cyclictest592687-21kworker/0:0+events20:31:220
54094499228,2cyclictest8287732kthreadcore23:56:322
54094499222,9cyclictest8558772unixbench_singl00:21:432
540944992220,1cyclictest557602-21kworker/u8:3+flush-8:021:01:302
540944992218,3cyclictest0-21swapper/219:55:002
540944992218,2cyclictest711326-21grep21:56:422
540944992218,2cyclictest697242-21aten2_r5power_c21:46:242
540944992218,2cyclictest0-21swapper/221:21:372
540944992217,4cyclictest646975-21unixbench_singl20:51:412
54093999225,16cyclictest592687-21kworker/0:0+events21:06:420
54094499213,13cyclictest821807-21apt-get23:51:232
54094499213,13cyclictest821807-21apt-get23:51:232
54094499212,13cyclictest662560-21apt-get21:11:222
540944992118,2cyclictest747047-21apt-config22:35:012
540944992115,4cyclictest0-21swapper/220:00:002
540944992115,2cyclictest0-21swapper/221:31:402
540944992115,2cyclictest0-21swapper/221:31:402
540939992117,2cyclictest0-21swapper/023:46:230
540939992116,3cyclictest0-21swapper/020:06:270
54094499208,4cyclictest0-21swapper/223:31:252
54094499208,4cyclictest0-21swapper/200:16:322
54094499201,4cyclictest821605-21wc23:50:002
540944992012,6cyclictest766797-21cat22:55:002
540944992011,7cyclictest715868-21grep22:01:372
54093999203,15cyclictest562574-21kworker/0:2+events19:52:180
540944991915,3cyclictest751659-21unixbench_singl22:36:422
540944991913,2cyclictest0-21swapper/223:41:402
54094499191,13cyclictest726856-21apt-get22:16:242
540939991914,2cyclictest0-21swapper/022:51:290
5404152194,9sleep20-21swapper/219:06:432
54094499188,7cyclictest639194-21tr20:46:302
54094499185,9cyclictest563121-21apt-get19:31:232
54094499185,9cyclictest563121-21apt-get19:31:232
540944991814,3cyclictest0-21swapper/223:20:002
540944991812,4cyclictest0-21swapper/220:11:292
5361372183,8sleep30-21swapper/319:05:323
5979592172,9sleep30-21swapper/320:06:243
540944991712,3cyclictest842608-21uname00:11:292
540944991712,3cyclictest0-21swapper/222:41:322
540944991711,2cyclictest0-21swapper/223:06:402
6731902166,6sleep30-21swapper/321:21:323
54094499162,13cyclictest0-21swapper/222:48:392
540944991613,2cyclictest0-21swapper/221:39:042
540944991611,4cyclictest767730-21irqcore22:56:292
54094499161,11cyclictest568897-21ps19:36:292
7820632146,5sleep30-21swapper/323:11:243
7618692142,7sleep30-21swapper/322:50:013
7268302142,8sleep30-21swapper/322:16:223
8605602132,7sleep30-21swapper/300:26:403
8313592131,8sleep30-21swapper/300:00:003
7109742132,7sleep30-21swapper/321:56:403
7018092132,6sleep30-21swapper/321:50:023
6411112132,7sleep30-21swapper/320:46:323
6178122132,6sleep30-21swapper/320:25:023
6025442132,6sleep30-21swapper/320:10:013
5927152132,6sleep30-21swapper/320:00:023
5868452132,7sleep30-21swapper/319:51:353
8700402122,6sleep30-21swapper/300:36:363
8408232122,6sleep30-21swapper/300:06:423
8318262122,6sleep30-21swapper/300:01:243
8318262122,6sleep30-21swapper/300:01:243
8172722123,6sleep30-21swapper/323:46:273
8116142122,5sleep30-21swapper/323:40:013
8057632122,6sleep30-21swapper/323:31:343
7721502122,6sleep30-21swapper/323:01:253
7662252122,6sleep30-21swapper/322:51:423
7520172122,5sleep30-21swapper/322:40:023
7165172122,6sleep30-21swapper/322:03:383
6976362122,6sleep30-21swapper/321:46:283
6910582123,5sleep30-21swapper/321:36:373
6773462122,6sleep30-21swapper/321:25:383
6522642122,5sleep30-21swapper/321:00:013
6516442122,6sleep30-21swapper/320:56:403
6277762122,6sleep30-21swapper/320:35:373
6124132121,7sleep30-21swapper/320:20:003
5734802122,6sleep30-21swapper/319:41:243
5721262122,6sleep30-21swapper/319:36:383
5526652122,6sleep30-21swapper/319:20:013
5526652122,6sleep30-21swapper/319:20:013
5480152122,6sleep30-21swapper/319:16:083
8514972112,5sleep30-21swapper/300:20:063
8480322112,5sleep30-21swapper/300:16:293
8417122113,5sleep30-21swapper/300:11:233
8220422112,6sleep30-21swapper/323:51:253
8220422112,6sleep30-21swapper/323:51:253
8003402112,5sleep30-21swapper/323:26:333
7927392112,5sleep30-21swapper/323:21:313
7909132112,6sleep30-21swapper/323:16:343
7769142112,5sleep30-21swapper/323:05:023
7674002112,5sleep30-21swapper/322:56:273
7472402112,5sleep30-21swapper/322:35:043
7424822112,5sleep30-21swapper/322:31:273
7371782112,5sleep30-21swapper/322:25:373
7218742112,6sleep30-21swapper/322:11:043
7168622112,5sleep30-21swapper/322:05:023
6674112112,5sleep30-21swapper/321:15:023
6666062112,5sleep30-21swapper/321:11:413
6578672112,5sleep30-21swapper/321:06:273
6425972113,5sleep30-21swapper/320:50:023
6330872112,5sleep30-21swapper/320:41:273
6232962112,6sleep30-21swapper/320:31:283
6130632112,5sleep30-21swapper/320:21:253
5883452112,5sleep30-21swapper/319:56:293
5781492112,5sleep30-21swapper/319:46:223
5583032112,5sleep30-21swapper/319:26:273
5583032112,5sleep30-21swapper/319:26:273
5455562112,5sleep30-21swapper/319:11:583
6921262102,5sleep30-21swapper/321:40:023
5629922102,5sleep30-21swapper/319:30:043
5629922102,5sleep30-21swapper/319:30:043
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional