You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-12 - 20:37

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot3s.osadl.org (updated Wed Aug 12, 2026 12:45:26)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
69560999793,2cyclictest0-21swapper/011:10:020
69560999692,2cyclictest0-21swapper/007:15:030
6956099862,15cyclictest80559-21munin-run09:20:020
6956099862,15cyclictest80559-21munin-run09:20:010
6956099861,2cyclictest0-21swapper/011:55:020
6956099851,3cyclictest0-21swapper/008:35:010
69560998435,44cyclictest16850irq/134-800000012:25:010
6956099842,41cyclictest117059-21ssh10:36:150
69560998328,50cyclictest114712-21ssh10:31:190
69560998026,14cyclictest119684-21ssh10:41:480
69560998025,12cyclictest165452-21ssh12:18:340
69560997925,12cyclictest146138-21ssh11:37:490
6956099783,4cyclictest104463-21munin-run10:10:000
69560997827,13cyclictest85095-21ssh09:29:310
69560997724,11cyclictest110969-21ssh10:23:350
69560997631,7cyclictest98194-21ssh09:56:330
69560997628,11cyclictest141417-21ssh11:27:540
69560997628,11cyclictest141417-21ssh11:27:540
6956099762,6cyclictest170957-21ls12:30:010
69560997625,12cyclictest96094-21ssh09:52:130
69560997625,12cyclictest96094-21ssh09:52:120
69560997625,12cyclictest158473-21ssh12:03:450
69560997625,11cyclictest91723-21ssh09:43:060
6956099762,39cyclictest173245-21ssh12:35:020
69560997528,11cyclictest149346-21ssh11:44:490
69560997528,11cyclictest126675-21ssh10:56:430
69560997527,13cyclictest108369-21ssh10:18:110
69560997427,10cyclictest100731-21ssh10:01:440
6956099742,65cyclictest151931-21ls11:50:010
69560997422,12cyclictest162199-21ssh12:11:280
69560997327,11cyclictest136102-21ssh11:16:290
69560997326,13cyclictest78061-21ssh09:14:510
69560997323,11cyclictest94480-21ssh09:48:590
69560997323,11cyclictest94480-21ssh09:48:580
69560997225,16cyclictest128967-21ssh11:01:300
69560997135,32cyclictest16950irq/224-800000007:40:000
69560997125,10cyclictest88977-21ssh09:37:290
69560997125,10cyclictest88977-21ssh09:37:290
69560997025,10cyclictest154469-21ssh11:55:070
69560997019,16cyclictest124211-21ssh10:51:230
6957099691,47cyclictest291rcuc/212:20:032
69560996927,8cyclictest87118-21ssh09:33:350
69560996923,11cyclictest82329-21ssh09:23:380
69560996819,13cyclictest133614-21ssh11:11:160
69560996818,17cyclictest16850irq/134-800000012:05:420
69574996718,10cyclictest37-21ksoftirqd/309:34:593
69570996535,8cyclictest142345-21kworker/2:411:55:542
69574996411,5cyclictest131rcu_preempt08:20:003
69570996343,12cyclictest68762-21kworker/2:011:03:182
6956599635,8cyclictest23-21ksoftirqd/109:10:021
6956599635,8cyclictest23-21ksoftirqd/109:10:011
69560996123,11cyclictest174800-21ssh12:38:160
69574996010,6cyclictest131rcu_preempt11:32:103
6956599603,11cyclictest23-21ksoftirqd/111:00:001
6956599601,9cyclictest175591-21munin-run12:40:011
6956599598,12cyclictest23-21ksoftirqd/111:30:021
6956599598,12cyclictest23-21ksoftirqd/111:30:021
6956599597,8cyclictest23-21ksoftirqd/110:35:011
6956099596,47cyclictest70059-21kworker/u8:407:25:010
69560995925,7cyclictest106839-21sed10:15:020
69560995916,17cyclictest16850irq/134-800000010:27:220
6957499588,6cyclictest131rcu_preempt12:35:013
69574995813,6cyclictest131rcu_preempt09:55:023
69574995813,6cyclictest131rcu_preempt09:55:013
69574995812,7cyclictest131rcu_preempt11:00:003
6956599587,8cyclictest23-21ksoftirqd/111:40:001
6956599587,8cyclictest23-21ksoftirqd/111:31:101
6956599587,10cyclictest23-21ksoftirqd/112:02:421
69565995813,12cyclictest23-21ksoftirqd/110:20:011
69570995748,5cyclictest142345-21kworker/2:412:04:362
6956599579,9cyclictest23-21ksoftirqd/109:24:341
6956599576,7cyclictest23-21ksoftirqd/109:40:021
6956599576,7cyclictest23-21ksoftirqd/109:40:011
69560995723,12cyclictest143626-21ssh11:32:370
6957499569,8cyclictest131rcu_preempt11:48:453
6957499562,5cyclictest131rcu_preempt10:35:023
69574995612,6cyclictest131rcu_preempt09:45:013
69570995648,5cyclictest70840-21kworker/2:110:01:322
69570995611,6cyclictest131rcu_preempt10:00:012
6956599566,9cyclictest23-21ksoftirqd/110:06:061
6956599566,9cyclictest23-21ksoftirqd/109:53:021
6956599566,9cyclictest23-21ksoftirqd/109:53:021
69565995614,6cyclictest131rcu_preempt09:49:501
69565995614,6cyclictest131rcu_preempt09:49:491
6957499557,6cyclictest131rcu_preempt11:10:023
6957499552,8cyclictest37-21ksoftirqd/312:10:023
69570995546,6cyclictest70840-21kworker/2:110:52:042
69570995544,6cyclictest70840-21kworker/2:110:33:182
69570995540,6cyclictest70840-21kworker/2:110:28:122
6956599559,5cyclictest131rcu_preempt10:50:011
6956599556,9cyclictest23-21ksoftirqd/110:42:071
69565995516,5cyclictest131rcu_preempt11:20:021
69565995513,10cyclictest23-21ksoftirqd/109:57:441
69565995512,6cyclictest131rcu_preempt09:14:271
69565995512,5cyclictest131rcu_preempt11:05:521
69565995511,6cyclictest131rcu_preempt09:26:541
69565995510,5cyclictest131rcu_preempt10:29:011
69560995516,16cyclictest16850irq/134-800000010:49:470
6957499547,11cyclictest37-21ksoftirqd/312:20:003
6957499542,7cyclictest37-21ksoftirqd/309:15:003
69570995446,5cyclictest142345-21kworker/2:411:43:452
69570995445,6cyclictest70840-21kworker/2:111:09:172
69570995444,6cyclictest142345-21kworker/2:412:31:362
69570995443,7cyclictest68762-21kworker/2:010:19:252
6956599546,10cyclictest23-21ksoftirqd/109:34:081
6956599543,7cyclictest23-21ksoftirqd/112:35:001
6956599542,10cyclictest23-21ksoftirqd/110:39:591
69565995410,4cyclictest131rcu_preempt12:30:031
6957499538,5cyclictest131rcu_preempt11:41:333
69570995345,5cyclictest139994-21kworker/2:212:12:482
69570995344,6cyclictest70840-21kworker/2:109:46:412
69570995344,6cyclictest70840-21kworker/2:109:46:402
69570995344,6cyclictest142345-21kworker/2:412:07:092
69570995344,5cyclictest68762-21kworker/2:009:16:202
69570995344,5cyclictest68762-21kworker/2:009:16:192
69570995343,6cyclictest68762-21kworker/2:009:40:362
69570995337,10cyclictest3467602-21systemd-journal08:50:012
6956599539,8cyclictest23-21ksoftirqd/108:50:001
6956599538,9cyclictest23-21ksoftirqd/112:08:291
6956599537,12cyclictest23-21ksoftirqd/111:57:431
6956599536,8cyclictest23-21ksoftirqd/112:15:011
6956599535,9cyclictest23-21ksoftirqd/110:20:171
69565995310,5cyclictest131rcu_preempt11:45:491
69560995331,14cyclictest3467602-21systemd-journal11:24:460
6957499526,11cyclictest37-21ksoftirqd/310:00:013
69574995212,6cyclictest131rcu_preempt10:05:013
69570995244,5cyclictest70840-21kworker/2:111:23:442
69570995244,5cyclictest70840-21kworker/2:111:15:562
69570995244,5cyclictest70840-21kworker/2:111:12:122
69570995244,5cyclictest68762-21kworker/2:009:14:132
69570995244,5cyclictest142345-21kworker/2:412:38:052
69570995244,5cyclictest142345-21kworker/2:412:29:472
69570995244,5cyclictest142345-21kworker/2:412:22:302
69570995243,6cyclictest68762-21kworker/2:009:37:502
69570995243,6cyclictest68762-21kworker/2:009:37:492
69570995243,5cyclictest70840-21kworker/2:110:06:012
69570995243,5cyclictest68762-21kworker/2:009:22:312
69570995242,6cyclictest68762-21kworker/2:010:58:292
69570995242,6cyclictest68762-21kworker/2:009:51:192
69570995242,6cyclictest68762-21kworker/2:009:51:192
6956599529,6cyclictest131rcu_preempt12:21:571
6956599529,6cyclictest131rcu_preempt10:50:091
6956599529,5cyclictest131rcu_preempt09:19:121
6956599529,5cyclictest131rcu_preempt09:19:111
6956599528,12cyclictest23-21ksoftirqd/107:35:021
6956599528,10cyclictest23-21ksoftirqd/111:05:021
6956599522,8cyclictest166086-21munin-run12:20:001
69565995212,9cyclictest131rcu_preempt11:14:161
69565995212,6cyclictest131rcu_preempt10:11:071
6956099522,44cyclictest75176-21kworker/u8:009:10:010
6956099522,44cyclictest75176-21kworker/u8:009:10:000
6957499519,5cyclictest131rcu_preempt11:28:163
6957499519,5cyclictest131rcu_preempt11:28:163
6957499512,5cyclictest131rcu_preempt11:40:003
6957499512,14cyclictest130588-21latency_hist11:05:013
69574995111,6cyclictest131rcu_preempt11:25:033
69574995111,6cyclictest131rcu_preempt07:30:003
69570995141,6cyclictest68762-21kworker/2:011:26:302
69570995141,6cyclictest68762-21kworker/2:011:26:302
69565995114,6cyclictest131rcu_preempt10:04:381
69565995114,5cyclictest131rcu_preempt11:53:331
69565995112,5cyclictest131rcu_preempt09:44:271
69565995112,5cyclictest131rcu_preempt07:30:021
69565995111,6cyclictest131rcu_preempt11:40:551
6957499507,5cyclictest131rcu_preempt10:16:123
6957499504,6cyclictest131rcu_preempt12:37:153
6957499502,8cyclictest37-21ksoftirqd/309:48:083
6957499502,8cyclictest37-21ksoftirqd/309:48:073
69574995024,15cyclictest37-21ksoftirqd/309:04:403
6957499502,16cyclictest37-21ksoftirqd/307:25:023
6957499501,4cyclictest131rcu_preempt10:50:003
69574995013,7cyclictest131rcu_preempt11:55:443
69574995011,7cyclictest131rcu_preempt10:10:363
69574995011,6cyclictest131rcu_preempt10:44:583
69570995043,4cyclictest142345-21kworker/2:411:50:532
69570995042,5cyclictest70840-21kworker/2:110:38:112
69570995042,5cyclictest68762-21kworker/2:010:47:302
69570995042,5cyclictest68762-21kworker/2:010:24:542
69570995042,5cyclictest68762-21kworker/2:009:25:022
69570995041,6cyclictest68762-21kworker/2:011:30:242
69570995041,6cyclictest68762-21kworker/2:010:11:392
69570995041,6cyclictest68762-21kworker/2:007:17:002
69570995036,10cyclictest68762-21kworker/2:009:30:362
69570995011,6cyclictest131rcu_preempt09:10:012
69570995011,6cyclictest131rcu_preempt09:10:002
6956599506,7cyclictest23-21ksoftirqd/107:49:591
69565995011,5cyclictest131rcu_preempt08:10:011
69560995036,10cyclictest1-21systemd07:35:020
6957499499,6cyclictest131rcu_preempt09:10:003
6957499499,6cyclictest131rcu_preempt09:10:003
6957499498,5cyclictest131rcu_preempt08:47:563
69574994933,12cyclictest3467602-21systemd-journal12:15:003
69574994920,10cyclictest37-21ksoftirqd/309:20:023
69574994920,10cyclictest37-21ksoftirqd/309:20:023
6957499491,7cyclictest37-21ksoftirqd/310:55:013
69574994913,9cyclictest73927-21/usr/sbin/munin08:35:003
69574994911,6cyclictest131rcu_preempt11:15:303
69574994911,6cyclictest131rcu_preempt09:24:003
69574994911,11cyclictest37-21ksoftirqd/309:38:013
69574994911,11cyclictest37-21ksoftirqd/309:38:003
69570994941,5cyclictest68762-21kworker/2:010:40:412
69570994941,5cyclictest142345-21kworker/2:411:45:512
69570994932,11cyclictest275-21dbus-daemon08:55:012
6956599498,9cyclictest23-21ksoftirqd/107:20:001
6956599496,9cyclictest23-21ksoftirqd/108:55:001
69565994913,6cyclictest131rcu_preempt07:25:011
69565994911,5cyclictest131rcu_preempt07:40:021
6956099496,29cyclictest69564-21latency_hist07:10:020
6956099491,4cyclictest131rcu_preempt08:49:590
6957499489,6cyclictest131rcu_preempt07:40:003
6957499488,5cyclictest131rcu_preempt12:24:343
6957499485,9cyclictest37-21ksoftirqd/308:30:023
6957499483,10cyclictest37-21ksoftirqd/312:05:003
69574994818,19cyclictest37-21ksoftirqd/307:13:403
69574994815,21cyclictest37-21ksoftirqd/308:12:543
69574994813,6cyclictest131rcu_preempt11:54:473
69570994840,5cyclictest68762-21kworker/2:008:18:522
69570994840,5cyclictest68762-21kworker/2:007:56:392
69570994840,5cyclictest68762-21kworker/2:007:56:382
69570994832,12cyclictest30-21ksoftirqd/207:30:012
69570994823,16cyclictest30-21ksoftirqd/211:40:012
69570994818,21cyclictest30-21ksoftirqd/209:05:012
6956599489,5cyclictest131rcu_preempt11:21:461
69565994837,7cyclictest23-21ksoftirqd/108:00:001
69565994837,7cyclictest23-21ksoftirqd/108:00:001
69565994813,5cyclictest131rcu_preempt08:40:011
69574994722,15cyclictest37-21ksoftirqd/307:34:073
6957499471,4cyclictest131rcu_preempt09:00:023
6957499471,4cyclictest131rcu_preempt09:00:013
69574994713,6cyclictest131rcu_preempt10:05:573
69574994710,6cyclictest131rcu_preempt08:10:003
69570994738,6cyclictest68762-21kworker/2:007:35:122
69570994735,10cyclictest30-21ksoftirqd/207:24:592
6956599479,5cyclictest131rcu_preempt07:15:021
69565994712,4cyclictest131rcu_preempt08:20:021
6956599471,11cyclictest23-21ksoftirqd/109:05:011
6957499463,9cyclictest37-21ksoftirqd/311:11:403
69574994634,8cyclictest37-21ksoftirqd/310:21:283
69574994631,11cyclictest3467602-21systemd-journal08:42:573
69574994615,6cyclictest131rcu_preempt10:25:533
69574994612,6cyclictest131rcu_preempt10:39:543
69570994639,4cyclictest68762-21kworker/2:008:31:592
69570994638,5cyclictest68762-21kworker/2:008:59:282
69570994638,5cyclictest68762-21kworker/2:008:59:272
69570994638,5cyclictest68762-21kworker/2:008:38:052
69570994638,5cyclictest68762-21kworker/2:008:28:552
69570994637,6cyclictest68762-21kworker/2:007:48:462
6956599463,8cyclictest23-21ksoftirqd/107:40:021
6956599462,9cyclictest23-21ksoftirqd/108:25:011
69565994629,12cyclictest275-21dbus-daemon08:35:011
6956099466,12cyclictest75222-21munin-plugin-st09:00:000
6956099466,12cyclictest75222-21munin-plugin-st09:00:000
69560994631,11cyclictest1-21systemd08:19:590
6957499456,7cyclictest37-21ksoftirqd/312:26:053
6957499456,5cyclictest131rcu_preempt08:25:003
6957499456,10cyclictest37-21ksoftirqd/309:29:123
6957499454,6cyclictest131rcu_preempt08:40:003
69570994538,4cyclictest68762-21kworker/2:008:11:062
69570994538,4cyclictest68762-21kworker/2:007:43:592
69570994537,5cyclictest68762-21kworker/2:008:21:062
6956599451,9cyclictest0-21swapper/108:13:431
69565994518,22cyclictest221rcuc/107:55:011
69560994537,5cyclictest16950irq/224-800000008:14:420
6957499442,9cyclictest37-21ksoftirqd/308:00:013
6957499442,9cyclictest37-21ksoftirqd/308:00:003
6957499441,4cyclictest131rcu_preempt07:50:023
69560994435,6cyclictest19350irq/34-mmc107:51:530
6956099443,36cyclictest69822-21cat07:15:010
6957499439,6cyclictest131rcu_preempt08:55:023
69570994335,5cyclictest68762-21kworker/2:007:51:592
6956599435,9cyclictest23-21ksoftirqd/109:00:011
6956599435,9cyclictest23-21ksoftirqd/109:00:001
69565994330,9cyclictest74455-21modprobe08:45:011
6956099435,32cyclictest70545-21cron07:30:000
69560994332,7cyclictest72084-21cron08:00:010
69560994332,7cyclictest72084-21cron08:00:000
69574994110,7cyclictest131rcu_preempt07:40:023
69570994133,6cyclictest68762-21kworker/2:008:41:362
69570994132,5cyclictest68762-21kworker/2:007:30:272
6956599412,9cyclictest23-21ksoftirqd/108:30:011
6956099413,6cyclictest0-21swapper/008:40:030
69560994133,6cyclictest19350irq/34-mmc107:45:020
69570994032,5cyclictest68762-21kworker/2:008:00:502
69570994032,5cyclictest68762-21kworker/2:007:10:172
6956099406,7cyclictest131rcu_preempt07:50:010
69560994033,3cyclictest12-21ksoftirqd/008:05:010
69560994032,5cyclictest19350irq/34-mmc108:42:410
69560994029,7cyclictest73660-21cron08:30:010
6957499392,6cyclictest131rcu_preempt07:10:023
69574993920,14cyclictest275-21dbus-daemon07:20:013
6957499391,4cyclictest131rcu_preempt07:50:383
69570993931,5cyclictest68762-21kworker/2:008:07:352
69560993927,7cyclictest309-21in:imuxsock09:05:010
6957499386,5cyclictest131rcu_preempt08:04:553
69565993825,8cyclictest23-21ksoftirqd/108:05:011
6956099386,22cyclictest3467602-21systemd-journal08:25:010
6956099372,31cyclictest3467602-21systemd-journal08:51:030
69560993627,6cyclictest0-21swapper/008:09:130
6956599341,7cyclictest23-21ksoftirqd/107:10:011
6957099321,5cyclictest131rcu_preempt07:10:012
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional