You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-21 - 21:07

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot3s.osadl.org (updated Sun Jun 21, 2026 12:45:33)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
135408997431,37cyclictest151rcuc/011:20:010
135408997431,37cyclictest151rcuc/011:20:000
13540899729,5cyclictest131rcu_preempt09:55:020
13540899723,10cyclictest121ksoftirqd/012:10:020
135413997162,6cyclictest23-21ksoftirqd/111:20:021
135413997162,6cyclictest23-21ksoftirqd/111:20:021
135408997032,18cyclictest12-21ksoftirqd/010:55:020
135408996933,31cyclictest12-21ksoftirqd/012:24:000
135408996915,38cyclictest12-21ksoftirqd/010:28:010
13540899689,5cyclictest131rcu_preempt09:39:500
135408996830,33cyclictest151rcuc/010:25:030
13541899672,10cyclictest37-21ksoftirqd/309:25:003
135415996742,7cyclictest169500-21kworker/2:211:49:182
13540899678,30cyclictest131rcu_preempt12:03:130
13541899646,7cyclictest37-21ksoftirqd/310:17:003
135418996417,8cyclictest37-21ksoftirqd/311:16:243
135418996417,8cyclictest37-21ksoftirqd/311:16:243
13540899644,28cyclictest131rcu_preempt09:30:190
13541899636,10cyclictest37-21ksoftirqd/308:25:013
13541899635,7cyclictest37-21ksoftirqd/309:13:063
13540899634,10cyclictest12-21ksoftirqd/010:10:000
135408996333,5cyclictest131rcu_preempt12:14:120
13540899627,5cyclictest131rcu_preempt07:35:000
135418996154,3cyclictest37-21ksoftirqd/311:35:023
135418996118,7cyclictest37-21ksoftirqd/311:57:513
135418996118,7cyclictest37-21ksoftirqd/311:57:503
135418996113,12cyclictest37-21ksoftirqd/312:35:033
13540899616,42cyclictest12-21ksoftirqd/011:00:300
13540899616,10cyclictest12-21ksoftirqd/010:00:010
135408996129,28cyclictest198743-21sh11:22:480
135408996115,5cyclictest131rcu_preempt07:20:000
13540899611,4cyclictest131rcu_preempt12:20:020
13541899607,9cyclictest37-21ksoftirqd/310:00:003
13541899604,9cyclictest37-21ksoftirqd/309:35:013
135418996011,12cyclictest37-21ksoftirqd/309:39:073
135415996039,13cyclictest141092-21kworker/2:010:16:242
135408996016,25cyclictest12-21ksoftirqd/010:47:460
135408996014,9cyclictest12-21ksoftirqd/010:31:170
13541899597,10cyclictest37-21ksoftirqd/311:15:003
13541899593,12cyclictest37-21ksoftirqd/309:10:003
13541899593,10cyclictest37-21ksoftirqd/311:55:003
135418995915,5cyclictest131rcu_preempt12:05:533
13540899593,8cyclictest12-21ksoftirqd/012:40:010
13540899593,8cyclictest12-21ksoftirqd/012:40:000
135408995922,11cyclictest12-21ksoftirqd/011:00:010
135408995912,6cyclictest131rcu_preempt08:10:000
135408995912,6cyclictest131rcu_preempt08:10:000
13541899588,7cyclictest37-21ksoftirqd/309:15:083
13541899588,7cyclictest37-21ksoftirqd/309:15:083
13541899583,8cyclictest37-21ksoftirqd/309:45:023
13541899583,17cyclictest37-21ksoftirqd/310:25:023
135418995819,9cyclictest131rcu_preempt12:27:103
135418995819,9cyclictest131rcu_preempt12:27:103
135418995814,4cyclictest131rcu_preempt12:13:033
135415995843,7cyclictest141092-21kworker/2:009:56:432
135415995843,6cyclictest169500-21kworker/2:210:56:502
13540899587,9cyclictest12-21ksoftirqd/011:10:020
13540899584,5cyclictest12-21ksoftirqd/012:32:430
135408995828,14cyclictest193771-21ssh11:11:120
13540899582,14cyclictest12-21ksoftirqd/011:30:010
13540899582,14cyclictest12-21ksoftirqd/011:30:010
135408995818,30cyclictest16850irq/134-800000009:42:300
135408995812,6cyclictest131rcu_preempt09:25:010
13541899577,10cyclictest37-21ksoftirqd/310:36:053
13541899576,7cyclictest37-21ksoftirqd/311:10:013
135418995714,6cyclictest131rcu_preempt10:10:523
135418995711,6cyclictest131rcu_preempt11:24:383
13541899571,15cyclictest37-21ksoftirqd/311:35:273
13541899571,15cyclictest37-21ksoftirqd/311:35:263
135418995710,12cyclictest37-21ksoftirqd/311:50:033
135415995742,7cyclictest141092-21kworker/2:010:47:422
135415995741,7cyclictest169500-21kworker/2:212:13:252
135408995711,7cyclictest131rcu_preempt09:45:130
13541899567,9cyclictest37-21ksoftirqd/312:21:393
13541899566,12cyclictest37-21ksoftirqd/310:29:473
135418995623,8cyclictest37-21ksoftirqd/310:59:483
135418995619,10cyclictest37-21ksoftirqd/308:30:003
135418995618,11cyclictest37-21ksoftirqd/310:42:273
13541899561,13cyclictest37-21ksoftirqd/310:50:003
135415995648,5cyclictest141092-21kworker/2:010:07:592
135415995639,6cyclictest169500-21kworker/2:211:44:422
135415995612,6cyclictest131rcu_preempt09:29:132
13541399562,37cyclictest141025-21kworker/u8:109:00:021
135408995619,16cyclictest16850irq/134-800000011:57:440
135408995619,16cyclictest16850irq/134-800000011:57:430
135408995616,9cyclictest12-21ksoftirqd/009:11:560
13541899558,5cyclictest131rcu_preempt11:04:183
13541899557,8cyclictest37-21ksoftirqd/308:10:023
13541899557,8cyclictest37-21ksoftirqd/308:10:023
13541899556,12cyclictest37-21ksoftirqd/311:29:093
13541899556,12cyclictest37-21ksoftirqd/311:29:093
13541899555,12cyclictest37-21ksoftirqd/309:05:023
13541899555,12cyclictest37-21ksoftirqd/309:05:023
135418995511,9cyclictest37-21ksoftirqd/310:00:533
135418995511,11cyclictest37-21ksoftirqd/308:55:013
135415995545,6cyclictest169500-21kworker/2:212:27:022
135415995545,6cyclictest169500-21kworker/2:212:27:012
135415995537,7cyclictest169500-21kworker/2:210:54:412
135408995528,10cyclictest12-21ksoftirqd/012:29:020
135408995528,10cyclictest12-21ksoftirqd/012:29:020
135408995527,4cyclictest131rcu_preempt10:37:120
135408995526,12cyclictest145032-21ssh09:17:540
135408995526,12cyclictest145032-21ssh09:17:540
135408995518,18cyclictest16850irq/134-800000009:29:510
135408995518,17cyclictest16850irq/134-800000010:10:070
135408995517,6cyclictest16850irq/134-800000010:42:440
13541899546,11cyclictest37-21ksoftirqd/309:51:013
13541899545,7cyclictest37-21ksoftirqd/310:07:233
13541899545,12cyclictest37-21ksoftirqd/308:15:013
13541899545,12cyclictest37-21ksoftirqd/308:15:013
135418995436,13cyclictest3467602-21systemd-journal09:50:013
135418995412,6cyclictest131rcu_preempt09:25:223
135418995410,6cyclictest131rcu_preempt12:36:503
135418995410,6cyclictest131rcu_preempt12:36:503
135415995412,5cyclictest131rcu_preempt09:25:002
13540899549,6cyclictest131rcu_preempt11:31:570
13540899544,5cyclictest131rcu_preempt08:05:020
135408995416,6cyclictest16850irq/134-800000011:47:160
135408995414,19cyclictest131rcu_preempt10:16:450
13541899535,14cyclictest37-21ksoftirqd/308:20:003
13541899534,9cyclictest37-21ksoftirqd/310:54:283
13541899534,8cyclictest37-21ksoftirqd/309:00:003
135418995338,11cyclictest140030-21systemctl08:40:013
13541899532,15cyclictest37-21ksoftirqd/308:05:003
135415995346,4cyclictest169500-21kworker/2:210:20:232
135415995344,5cyclictest169500-21kworker/2:211:32:232
135415995339,6cyclictest169500-21kworker/2:211:37:562
135415995339,6cyclictest169500-21kworker/2:211:37:562
135408995327,11cyclictest204314-21ssh11:35:360
135408995327,11cyclictest204314-21ssh11:35:360
13541899526,9cyclictest37-21ksoftirqd/308:45:023
13541899526,9cyclictest37-21ksoftirqd/307:15:023
13541899526,10cyclictest37-21ksoftirqd/311:40:303
13541899523,10cyclictest37-21ksoftirqd/310:32:293
13541899521,13cyclictest37-21ksoftirqd/312:15:083
13541599527,6cyclictest131rcu_preempt08:10:002
13541599527,6cyclictest131rcu_preempt08:10:002
13541599526,6cyclictest131rcu_preempt09:10:022
135415995244,5cyclictest143647-21kworker/2:109:35:352
135415995244,5cyclictest141092-21kworker/2:009:17:272
135415995244,5cyclictest141092-21kworker/2:009:17:262
135415995243,5cyclictest141092-21kworker/2:010:14:202
135415995237,7cyclictest169500-21kworker/2:212:07:232
135415995234,6cyclictest169500-21kworker/2:212:01:012
13540899528,8cyclictest12-21ksoftirqd/008:15:010
13540899528,8cyclictest12-21ksoftirqd/008:15:010
135408995219,29cyclictest12-21ksoftirqd/011:54:470
135408995215,6cyclictest16850irq/134-800000011:44:500
135408995212,5cyclictest131rcu_preempt10:00:350
135408995211,6cyclictest131rcu_preempt09:10:000
135408995211,6cyclictest131rcu_preempt08:45:020
13541899517,9cyclictest37-21ksoftirqd/307:35:003
13541899517,9cyclictest37-21ksoftirqd/307:30:013
13541899517,9cyclictest37-21ksoftirqd/307:30:013
13541899511,13cyclictest37-21ksoftirqd/307:40:003
135418995110,12cyclictest211501-1kworker/u9:112:05:003
135415995143,5cyclictest169500-21kworker/2:212:30:342
135415995143,5cyclictest169500-21kworker/2:212:19:522
135415995143,5cyclictest169500-21kworker/2:211:23:272
135415995143,5cyclictest169500-21kworker/2:210:34:572
135415995143,5cyclictest169500-21kworker/2:210:26:422
135415995142,6cyclictest134619-21kworker/2:107:30:012
135415995142,6cyclictest134619-21kworker/2:107:30:012
135415995141,5cyclictest134619-21kworker/2:108:11:032
135415995141,5cyclictest134619-21kworker/2:108:11:032
135415995134,7cyclictest206048-21kworker/2:312:36:312
135415995134,7cyclictest206048-21kworker/2:312:36:312
13541399518,11cyclictest131rcu_preempt10:40:001
13540899519,6cyclictest131rcu_preempt08:55:010
13540899513,7cyclictest12-21ksoftirqd/008:50:020
13540899513,7cyclictest12-21ksoftirqd/008:50:020
135408995112,6cyclictest131rcu_preempt09:00:010
13541899504,11cyclictest37-21ksoftirqd/307:50:003
13541899501,14cyclictest37-21ksoftirqd/308:35:013
135415995043,4cyclictest134619-21kworker/2:107:30:132
135415995042,5cyclictest206048-21kworker/2:311:54:492
135415995042,5cyclictest143647-21kworker/2:109:31:102
135415995042,5cyclictest141092-21kworker/2:011:10:452
135415995042,5cyclictest141092-21kworker/2:010:44:402
135415995042,5cyclictest141092-21kworker/2:009:50:242
135415995042,5cyclictest141092-21kworker/2:009:41:402
135415995042,5cyclictest141092-21kworker/2:009:11:402
135415995041,5cyclictest206048-21kworker/2:312:22:082
135415995014,6cyclictest131rcu_preempt08:30:002
135415995011,9cyclictest131rcu_preempt10:03:382
13540899507,9cyclictest12-21ksoftirqd/007:30:010
13540899507,9cyclictest12-21ksoftirqd/007:30:010
13540899502,13cyclictest140018-21cron08:40:000
13540899501,45cyclictest0-21swapper/007:40:020
13541899496,10cyclictest37-21ksoftirqd/308:00:013
135415994941,5cyclictest206048-21kworker/2:311:58:242
135415994941,5cyclictest206048-21kworker/2:311:58:242
135415994941,5cyclictest169500-21kworker/2:211:25:042
135415994941,5cyclictest169500-21kworker/2:211:25:032
135415994941,5cyclictest169500-21kworker/2:210:37:332
135415994941,5cyclictest141092-21kworker/2:011:01:282
135415994941,5cyclictest141092-21kworker/2:009:45:562
135415994940,6cyclictest169500-21kworker/2:211:15:062
135415994940,6cyclictest169500-21kworker/2:211:15:052
135415994938,4cyclictest141092-21kworker/2:011:07:262
13541599492,5cyclictest131rcu_preempt07:40:002
135413994939,6cyclictest136641-21munin-plugin-st07:35:001
135413994910,6cyclictest131rcu_preempt11:55:011
135408994910,35cyclictest12-21ksoftirqd/007:15:010
13541899486,9cyclictest37-21ksoftirqd/307:20:013
13541899484,7cyclictest37-21ksoftirqd/307:55:033
13541899484,10cyclictest37-21ksoftirqd/307:25:013
135415994841,4cyclictest134619-21kworker/2:107:44:342
135415994840,5cyclictest139036-21kworker/2:208:43:022
13541399484,6cyclictest23-21ksoftirqd/112:14:041
135413994833,11cyclictest1-21systemd07:40:011
13541399481,7cyclictest206086-21grep11:40:021
13541399481,7cyclictest206086-21grep11:40:021
13540899489,10cyclictest12-21ksoftirqd/009:05:020
13540899489,10cyclictest12-21ksoftirqd/009:05:020
13540899487,5cyclictest131rcu_preempt07:50:020
135408994810,9cyclictest131rcu_preempt07:45:020
13541899474,10cyclictest37-21ksoftirqd/307:10:023
13541899473,13cyclictest37-21ksoftirqd/308:50:023
13541899473,13cyclictest37-21ksoftirqd/308:50:013
135415994740,4cyclictest139036-21kworker/2:208:51:292
135415994739,5cyclictest134619-21kworker/2:107:46:192
135415994739,5cyclictest134619-21kworker/2:107:20:092
135415994722,22cyclictest30-21ksoftirqd/208:40:012
135413994724,5cyclictest23-21ksoftirqd/109:11:101
135413994724,16cyclictest275-21dbus-daemon10:00:011
135413994717,9cyclictest23-21ksoftirqd/109:40:021
135413994714,8cyclictest131rcu_preempt09:25:011
135413994710,17cyclictest131rcu_preempt10:35:021
13540899471,4cyclictest131rcu_preempt08:24:170
13541899461,15cyclictest37-21ksoftirqd/307:45:023
135415994638,5cyclictest134619-21kworker/2:107:50:232
135415994638,5cyclictest134619-21kworker/2:107:16:342
135415994637,6cyclictest141092-21kworker/2:009:00:232
135415994637,6cyclictest141092-21kworker/2:009:00:232
135415994628,14cyclictest3467602-21systemd-journal08:05:022
13541599461,7cyclictest140690-21latency_hist08:50:022
13541599461,7cyclictest140690-21latency_hist08:50:022
135413994610,5cyclictest131rcu_preempt11:27:381
135413994610,5cyclictest131rcu_preempt11:27:381
13540899468,6cyclictest131rcu_preempt08:30:010
13540899466,7cyclictest12-21ksoftirqd/008:00:020
135415994537,5cyclictest134619-21kworker/2:107:55:242
135415994537,5cyclictest134619-21kworker/2:107:12:062
135413994532,9cyclictest1-21systemd11:05:011
135413994528,13cyclictest275-21dbus-daemon11:25:011
13540899457,8cyclictest12-21ksoftirqd/008:20:000
13540899455,7cyclictest12-21ksoftirqd/007:55:000
135408994511,5cyclictest131rcu_preempt08:35:020
135415994437,4cyclictest139036-21kworker/2:208:58:062
135415994430,10cyclictest1-21systemd08:20:022
13541599441,10cyclictest311-21rs:main2
135413994430,11cyclictest3467602-21systemd-journal08:36:321
135413994429,11cyclictest224316-21sshd12:22:231
13541399442,5cyclictest131rcu_preempt07:30:011
13541399442,5cyclictest131rcu_preempt07:30:011
135408994410,11cyclictest12-21ksoftirqd/007:23:000
135415994335,5cyclictest139036-21kworker/2:208:31:152
13541399434,4cyclictest131rcu_preempt10:20:511
13541399433,5cyclictest131rcu_preempt08:35:011
135413994330,9cyclictest309-21in:imuxsock11:55:031
135413994330,9cyclictest309-21in:imuxsock11:55:021
135413994328,11cyclictest23-21ksoftirqd/112:16:231
135413994325,9cyclictest23-21ksoftirqd/109:05:011
135413994325,9cyclictest23-21ksoftirqd/109:05:011
135413994320,9cyclictest131rcu_preempt11:15:031
13541399431,5cyclictest131rcu_preempt09:20:001
13541399431,5cyclictest131rcu_preempt09:20:001
13541399431,4cyclictest0-21swapper/109:50:231
135413994314,5cyclictest131rcu_preempt11:34:031
135413994227,11cyclictest275-21dbus-daemon10:25:481
135413994226,12cyclictest275-21dbus-daemon12:35:221
135413994226,12cyclictest275-21dbus-daemon12:35:221
135413994226,12cyclictest1-21systemd08:50:011
135413994226,12cyclictest1-21systemd08:50:011
13541399422,5cyclictest131rcu_preempt10:07:001
135413994218,14cyclictest23-21ksoftirqd/107:17:311
13541399421,5cyclictest131rcu_preempt09:34:591
135413994215,17cyclictest23-21ksoftirqd/110:15:031
135413994213,7cyclictest131rcu_preempt07:10:021
135413994213,5cyclictest131rcu_preempt11:00:021
135413994212,12cyclictest23-21ksoftirqd/110:05:011
13541399412,5cyclictest131rcu_preempt11:49:361
13541399411,7cyclictest0-21swapper/109:25:361
13541399411,3cyclictest131rcu_preempt10:20:031
13541399409,5cyclictest131rcu_preempt12:35:001
13541399407,25cyclictest131rcu_preempt10:51:411
13541399403,6cyclictest131rcu_preempt12:05:001
13541399403,5cyclictest131rcu_preempt11:08:201
135413994031,6cyclictest138966-21cron08:20:001
13541399402,5cyclictest131rcu_preempt09:46:441
135413994021,14cyclictest275-21dbus-daemon07:55:011
135413994016,15cyclictest23-21ksoftirqd/107:20:441
135413994014,5cyclictest131rcu_preempt10:40:441
135413994012,25cyclictest131rcu_preempt12:26:551
135413994012,25cyclictest131rcu_preempt12:26:551
13541399396,7cyclictest131rcu_preempt08:30:011
135413993927,10cyclictest23-21ksoftirqd/109:41:391
13541399392,5cyclictest131rcu_preempt09:10:011
13541399391,4cyclictest131rcu_preempt11:41:141
13541399391,3cyclictest131rcu_preempt08:05:001
13541599385,5cyclictest131rcu_preempt07:10:022
135413993823,11cyclictest3467602-21systemd-journal08:10:011
135413993823,11cyclictest3467602-21systemd-journal08:10:011
135413993821,6cyclictest131rcu_preempt10:45:461
135413993820,9cyclictest23-21ksoftirqd/108:55:011
13541399381,4cyclictest131rcu_preempt12:05:161
135413993726,9cyclictest231ksoftirqd/108:00:011
135413993619,6cyclictest131rcu_preempt08:25:011
135413993619,13cyclictest1-21systemd08:45:011
135413993617,15cyclictest131rcu_preempt08:15:011
135413993617,15cyclictest131rcu_preempt08:15:011
135413993615,14cyclictest23-21ksoftirqd/107:43:341
13540899353,7cyclictest12-21ksoftirqd/007:10:010
13541399345,5cyclictest131rcu_preempt07:14:021
135413993320,10cyclictest23-21ksoftirqd/107:48:311
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional