You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-11 - 22:49

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: ondemand
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot4s.osadl.org (updated Tue Aug 11, 2026 12:44:13)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
29598599200179,19cyclictest481662-21munin-run10:15:010
52587321920,4sleep30-21swapper/310:52:543
2959929910196,3cyclictest481714-21apt-config10:15:012
3663772870,3sleep3575-21kworker/3:2-events08:17:543
4637892860,5sleep30-21swapper/309:57:533
295989998683,2cyclictest0-21swapper/110:15:011
4505762610,2sleep3481ktimers/309:42:573
295989995852,4cyclictest441840-21kworker/u8:1+flush-8:009:43:061
295992995550,4cyclictest596685-21kworker/u8:1+events_unbound12:03:042
295992995448,4cyclictest596685-21kworker/u8:1+events_unbound12:22:502
295985995432,7cyclictest532705-21/usr/sbin/munin11:02:560
295989995144,5cyclictest491711-21kworker/u8:4+flush-8:010:32:571
295992995046,3cyclictest461086-21kworker/u8:0+events_unbound10:00:002
295989994943,4cyclictest466603-21kworker/u8:2+events_unbound10:22:491
295989994941,6cyclictest622497-21/usr/sbin/munin12:33:021
295992994844,3cyclictest491711-21kworker/u8:4+events_unbound11:15:012
295989994843,4cyclictest427158-21kworker/u8:2+flush-8:009:48:001
295989994842,4cyclictest532109-21kworker/u8:3+events_unbound11:12:511
295989994742,4cyclictest530744-21kworker/u8:2+events_unbound11:35:001
295989994742,4cyclictest3694173-21kworker/u8:4+events_unbound08:33:061
295989994740,5cyclictest596685-21kworker/u8:1+flush-8:012:18:131
295989994740,5cyclictest427158-21kworker/u8:2+flush-8:009:52:581
295989994640,4cyclictest461086-21kworker/u8:0+flush-8:010:18:431
295992994541,3cyclictest491711-21kworker/u8:4+events_unbound10:52:582
295992994439,4cyclictest596685-21kworker/u8:1+events_unbound12:38:032
295989994440,3cyclictest582476-21kworker/u8:3+events_unbound11:57:501
295989994437,5cyclictest572364-21kworker/u8:0+flush-8:011:52:491
295989994338,4cyclictest532109-21kworker/u8:3+events_unbound11:27:461
295989994338,4cyclictest532109-21kworker/u8:3+events_unbound11:27:461
295989994338,4cyclictest367244-21kworker/u8:1+flush-8:008:23:021
4939022420,3sleep3575-21kworker/3:2+events10:22:533
295992994237,4cyclictest530744-21kworker/u8:2+events_unbound10:57:552
295989994237,3cyclictest507259-21kworker/u8:0+events_unbound10:50:001
295992994135,4cyclictest3694173-21kworker/u8:4+events_unbound08:27:512
295989994137,3cyclictest347206-21kworker/u8:0+events_unbound09:32:451
295989994135,5cyclictest347206-21kworker/u8:0+events_unbound08:37:471
295989994135,4cyclictest530744-21kworker/u8:2+events_unbound10:57:571
295989994134,5cyclictest530744-21kworker/u8:2+flush-8:012:03:001
295989994134,5cyclictest470957-21ntpq10:02:571
295989994035,3cyclictest466603-21kworker/u8:2+events_unbound10:27:511
295989994035,3cyclictest347206-21kworker/u8:0+flush-8:009:12:511
295989994034,4cyclictest3694173-21kworker/u8:4+events_unbound07:22:511
295985994030,8cyclictest275-21systemd-journal11:55:000
295989993934,3cyclictest530744-21kworker/u8:2+events_unbound11:43:001
2913812394,32sleep20-21swapper/207:07:322
295989993833,4cyclictest3694173-21kworker/u8:4+events_unbound07:15:001
295989993832,4cyclictest532109-21kworker/u8:3+events_unbound11:07:491
295989993832,4cyclictest312984-21kworker/u8:2+flush-8:008:02:551
29598599382,34cyclictest357458-21kworker/0:0+events08:33:020
295992993733,3cyclictest436368-21kworker/u8:3+events_unbound09:28:022
295989993733,3cyclictest491711-21kworker/u8:4+events_unbound11:02:491
295989993732,4cyclictest596685-21kworker/u8:1+flush-8:012:27:471
295989993732,4cyclictest530744-21kworker/u8:2+flush-8:011:47:481
295989993731,4cyclictest530744-21kworker/u8:2+flush-8:012:07:581
295989993728,7cyclictest556612-21ntpq11:22:591
295985993725,9cyclictest551555-21ntpq11:17:560
295992993632,3cyclictest312984-21kworker/u8:2+events_unbound08:15:002
295989993632,3cyclictest3694173-21kworker/u8:4+events_unbound07:42:541
295989993631,4cyclictest507259-21kworker/u8:0+flush-8:010:37:581
295989993625,8cyclictest347383-21awk08:00:001
295992993530,3cyclictest596685-21kworker/u8:1+events_unbound12:10:382
295989993531,3cyclictest441840-21kworker/u8:1+events_unbound09:39:071
295989993531,3cyclictest347206-21kworker/u8:0+events_unbound08:48:011
295989993531,3cyclictest347206-21kworker/u8:0+events_unbound08:48:011
295989993529,4cyclictest507259-21kworker/u8:0+events_unbound10:52:521
295989993529,4cyclictest3694173-21kworker/u8:4+events_unbound08:27:471
295985993524,9cyclictest490893-21mandb10:18:450
295989993429,3cyclictest402351-21kworker/u8:1+flush-8:009:07:541
295989993429,3cyclictest3682258-21kworker/u8:1+events_unbound07:52:521
295989993428,5cyclictest312984-21kworker/u8:2+flush-8:007:32:491
295985993425,7cyclictest507457-21/usr/sbin/munin10:38:020
295985993424,7cyclictest572325-21latency_hist11:40:010
29598599341,28cyclictest357458-21kworker/0:0+events08:52:420
295992993326,5cyclictest3682258-21kworker/u8:1+events_unbound07:30:042
295989993329,3cyclictest596685-21kworker/u8:1+events_unbound12:22:521
295989993329,2cyclictest367392-21cron08:20:001
295989993328,4cyclictest596685-21kworker/u8:1+flush-8:012:13:051
295989993328,4cyclictest436368-21kworker/u8:3+flush-8:010:08:031
295985993326,5cyclictest392961-21grep08:47:470
295985993326,5cyclictest392961-21grep08:47:470
295985993310,19cyclictest295864-21kworker/0:2+events07:46:540
295989993228,3cyclictest347206-21kworker/u8:0+flush-8:009:28:031
295989993227,3cyclictest567234-21kworker/u8:1+flush-8:011:37:531
29598599329,17cyclictest357458-21kworker/0:0+events10:02:460
29598599328,22cyclictest328586-21kworker/0:1+events07:42:540
295985993225,5cyclictest627016-21latency_hist12:35:000
295985993224,6cyclictest327611-21latency_hist07:40:010
295985993222,8cyclictest596650-21ntpq12:03:010
29598599322,28cyclictest295864-21kworker/0:2+events07:22:580
29598599321,26cyclictest537422-21kworker/0:2+events12:40:000
295985993212,18cyclictest537422-21kworker/0:2+events11:31:500
295992993126,4cyclictest466603-21kworker/u8:2+events_unbound10:20:032
295989993125,4cyclictest347206-21kworker/u8:0+flush-8:008:52:561
295989993124,5cyclictest352820-21apt-get08:07:431
295985993125,4cyclictest367882-21apt-get08:22:430
29598599312,27cyclictest357458-21kworker/0:0+events09:37:480
295992993026,3cyclictest530744-21kworker/u8:2+flush-8:011:03:192
295992993026,3cyclictest3694173-21kworker/u8:4+events_unbound08:33:082
295992993026,3cyclictest347206-21kworker/u8:0+flush-8:008:55:052
295992993025,3cyclictest275-21systemd-journal09:05:002
295992993024,4cyclictest532109-21kworker/u8:3+events_unbound11:30:202
295989993026,2cyclictest464417-21kthreadcore09:57:541
295989993024,5cyclictest3694173-21kworker/u8:4+flush-8:007:37:491
295989993024,5cyclictest3694173-21kworker/u8:4+events_unbound07:47:551
295989993023,5cyclictest507259-21kworker/u8:0+events_unbound10:40:021
295989993023,5cyclictest491711-21kworker/u8:4+flush-8:011:18:041
29598599309,19cyclictest347402-21kworker/0:1+events08:10:000
29598599307,19cyclictest357458-21kworker/0:0+events09:57:480
29598599305,23cyclictest357458-21kworker/0:0+events10:47:560
295985993027,2cyclictest456851-21idleruntime-cro09:50:010
295985993024,4cyclictest303518-21fschecks_count07:17:470
295985993020,8cyclictest576599-21ntpq11:42:570
295985993010,18cyclictest537422-21kworker/0:2+events11:08:010
295992992925,3cyclictest438206-21grep09:32:502
295992992925,3cyclictest401-21dbus-daemon08:40:002
295992992922,5cyclictest431958-21cp09:25:002
295989992924,3cyclictest387460-21kworker/u8:2+flush-8:008:42:521
295989992923,4cyclictest417192-21kworker/u8:4+flush-8:009:18:041
295989992919,8cyclictest406311-21ntpq08:57:561
29598599295,22cyclictest357458-21kworker/0:0+events10:23:000
29598599293,24cyclictest357458-21kworker/0:0+events08:12:520
295985992923,4cyclictest3694173-21kworker/u8:4+flush-8:007:13:000
295985992922,5cyclictest532109-21kworker/u8:3+flush-8:011:12:530
29598599291,26cyclictest537422-21kworker/0:2+events11:27:520
29598599291,26cyclictest537422-21kworker/0:2+events11:27:520
29598599291,26cyclictest295864-21kworker/0:2+events07:58:040
295992992824,3cyclictest507259-21kworker/u8:0+events_unbound10:48:042
295992992823,3cyclictest397957-21fschecks_time08:52:472
295992992820,6cyclictest275-21systemd-journal10:45:002
295989992823,4cyclictest312984-21kworker/u8:2+flush-8:007:27:591
29598599287,19cyclictest357458-21kworker/0:0+events09:29:140
29598599282,25cyclictest357458-21kworker/0:0+events09:23:040
29598599281,26cyclictest357458-21kworker/0:0+events10:30:000
29598599281,25cyclictest537422-21kworker/0:2+events12:12:560
295985992810,16cyclictest357458-21kworker/0:0+events09:32:520
295985992810,16cyclictest357458-21kworker/0:0+events08:27:560
29598599280,26cyclictest537422-21kworker/0:2+events11:57:540
295992992724,2cyclictest482647-21cut10:17:472
295992992723,3cyclictest3694173-21kworker/u8:4+events_unbound08:18:052
295989992723,3cyclictest263430-21kworker/u8:0+flush-8:007:18:041
295989992720,5cyclictest596685-21kworker/u8:1+events_unbound12:37:541
295989992719,6cyclictest361858-21ntpq08:13:001
29598599278,17cyclictest295864-21kworker/0:2+events07:52:500
29598599277,18cyclictest295864-21kworker/0:2+events07:30:000
295985992723,2cyclictest0-21swapper/009:20:000
295985992720,5cyclictest451153-21ntpq09:42:590
295985992719,4cyclictest0-21swapper/007:32:450
5267262260,1sleep0526729-21sort10:53:030
295992992622,3cyclictest497529-21cpuspeed_turbos10:27:462
295989992618,6cyclictest347206-21kworker/u8:0+flush-8:009:02:581
29598599268,16cyclictest537422-21kworker/0:2+events12:27:580
29598599266,19cyclictest357458-21kworker/0:0+events10:59:340
29598599262,23cyclictest0-21swapper/012:08:030
29598599261,23cyclictest537422-21kworker/0:2+events11:47:560
295985992610,14cyclictest357458-21kworker/0:0+events10:32:440
2958172262,6sleep00-21swapper/007:09:550
4143362250,7sleep3451irq_work/309:07:543
295992992522,2cyclictest422441-21aten2_r5power_e09:17:462
295989992520,3cyclictest0-21swapper/109:25:001
29598599256,18cyclictest357458-21kworker/0:0+events09:52:520
29598599255,18cyclictest295864-21kworker/0:2+events08:02:470
295985992522,2cyclictest0-21swapper/009:10:000
295985992518,5cyclictest367165-21grep08:18:040
295985992517,6cyclictest512715-21apt-get10:42:440
295985992515,8cyclictest475993-21ntpq10:08:000
5780212240,6sleep3481ktimers/311:47:473
295992992420,2cyclictest447489-21aten2_r5power_e09:42:482
295992992419,4cyclictest312984-21kworker/u8:2+flush-8:008:01:072
29598599249,14cyclictest537422-21kworker/0:2+events11:23:350
29598599248,14cyclictest537422-21kworker/0:2+events12:16:180
29598599246,16cyclictest357458-21kworker/0:0+events08:37:440
29598599243,20cyclictest357458-21kworker/0:0+events09:00:080
29599299232,4cyclictest0-21swapper/210:07:452
295992992319,3cyclictest582476-21kworker/u8:3+events_unbound11:50:332
295992992319,3cyclictest572364-21kworker/u8:0+events_unbound11:46:552
295992992319,3cyclictest491711-21kworker/u8:4+events_unbound11:15:062
295992992319,2cyclictest342191-21unixbench_multi07:53:052
295992992318,3cyclictest573119-21fschecks_time11:42:482
295985992320,2cyclictest392180-21cron08:45:000
295985992317,4cyclictest403872-21kthreadcore08:57:520
29599299222,4cyclictest0-21swapper/212:17:532
295992992219,2cyclictest357096-21unixbench_singl08:08:052
295992992218,3cyclictest530744-21kworker/u8:2+events_unbound12:26:072
295992992218,3cyclictest296447-21aten2_r5power_p07:12:452
295992992217,4cyclictest3694173-21kworker/u8:4+events_unbound07:56:092
29598599223,17cyclictest357458-21kworker/0:0+events09:12:440
295985992211,3cyclictest0-21swapper/012:22:460
295992992117,3cyclictest461086-21kworker/u8:0+events_unbound09:53:212
295992992116,4cyclictest622080-21kworker/u8:0+flush-8:012:30:212
29599299211,10cyclictest3123930-21kworker/2:2+events11:37:532
2911932214,11sleep10-21swapper/107:05:061
29599299202,13cyclictest391526-21ntpq08:42:582
295992992016,2cyclictest442766-21fschecks_time09:37:492
295992992016,2cyclictest303568-21fschecks_time07:17:472
29599299200,5cyclictest592374-21munin-run12:00:002
295992991916,2cyclictest452603-21fschecks_count09:47:462
295992991915,2cyclictest0-21swapper/211:25:002
295992991915,2cyclictest0-21swapper/210:40:012
295992991914,3cyclictest327356-21unixbench_multi07:38:042
295992991913,1cyclictest161rcu_preempt07:42:562
295992991910,7cyclictest419977-21tr09:12:532
2912402193,9sleep30-21swapper/307:05:443
29599299184,2cyclictest421rcuog/207:22:582
295992991814,2cyclictest412795-21cut09:07:492
29599299179,6cyclictest469451-21tr10:02:542
29599299177,8cyclictest504412-21tr10:32:542
29599299173,6cyclictest0-21swapper/208:23:002
29599299172,5cyclictest0-21swapper/212:05:342
295992991713,2cyclictest0-21swapper/211:28:012
295992991713,2cyclictest0-21swapper/211:28:012
295992991711,4cyclictest0-21swapper/211:07:542
295992991710,5cyclictest313216-21seq07:27:432
29599299170,2cyclictest421rcuog/208:47:522
29599299170,2cyclictest421rcuog/208:47:522
29599299166,8cyclictest371rcuc/207:47:492
3339552150,4sleep3575-21kworker/3:2-mm_percpu_wq07:47:493
3317322150,2sleep3481ktimers/307:42:553
5633232140,3sleep3481ktimers/311:32:513
5723912132,6sleep30-21swapper/311:40:023
5579882132,7sleep30-21swapper/311:27:463
5579882132,7sleep30-21swapper/311:27:463
4672812132,7sleep30-21swapper/310:02:483
4112012131,8sleep30-21swapper/309:02:573
4033072134,6sleep30-21swapper/308:57:523
3785782132,7sleep30-21swapper/308:32:563
3600632132,7sleep30-21swapper/308:12:543
3474042132,6sleep30-21swapper/308:00:013
3140432132,7sleep30-21swapper/307:27:513
6273842122,6sleep30-21swapper/312:36:523
6179382122,6sleep30-21swapper/312:27:503
6071962122,5sleep30-21swapper/312:15:013
6038952122,6sleep30-21swapper/312:12:563
5974602122,5sleep30-21swapper/312:05:023
5878692122,6sleep30-21swapper/311:57:443
5864472122,6sleep30-21swapper/311:52:583
5442672122,6sleep30-21swapper/311:12:543
5419822122,6sleep30-21swapper/311:08:043
5323342122,6sleep30-21swapper/311:00:023
5215522122,6sleep30-21swapper/310:48:023
5171752120,6sleep30-21swapper/310:45:013
4981912122,6sleep30-21swapper/310:27:523
4823012122,6sleep30-21swapper/310:17:443
4721472122,6sleep30-21swapper/310:07:453
4536642123,5sleep30-21swapper/309:47:513
4462882123,6sleep30-21swapper/309:38:003
4370562122,6sleep30-21swapper/309:30:023
4261472123,6sleep30-21swapper/309:17:583
4217002122,6sleep30-21swapper/309:13:043
3971712122,6sleep30-21swapper/308:50:013
3926522122,7sleep30-21swapper/308:47:443
3926522122,7sleep30-21swapper/308:47:443
3724282122,5sleep30-21swapper/308:25:013
3077972122,5sleep30-21swapper/307:20:023
3013782122,6sleep30-21swapper/307:15:033
6222422112,5sleep30-21swapper/312:30:023
6126002112,5sleep30-21swapper/312:22:433
5926492112,5sleep30-21swapper/312:00:323
5722982112,6sleep30-21swapper/311:39:503
5526462113,5sleep30-21swapper/311:21:323
5475952112,5sleep30-21swapper/311:15:023
5275392112,5sleep30-21swapper/310:55:043
5069592111,5sleep30-21swapper/310:35:023
5031752112,5sleep30-21swapper/310:32:533
4766972112,5sleep30-21swapper/310:10:023
4605072112,5sleep30-21swapper/309:52:573
4323262112,5sleep30-21swapper/309:25:083
4317432112,6sleep30-21swapper/309:23:403
3876642112,5sleep30-21swapper/308:42:433
3864972112,5sleep30-21swapper/308:37:573
3677252112,5sleep30-21swapper/308:21:043
3527942112,5sleep30-21swapper/308:07:443
3426722112,6sleep30-21swapper/307:55:043
3379552112,5sleep30-21swapper/307:52:473
3228832112,5sleep30-21swapper/307:35:343
3179152112,6sleep30-21swapper/307:30:043
2961252112,5sleep30-21swapper/307:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional