You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-18 - 16:33

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot3s.osadl.org (updated Wed Mar 18, 2026 12:45:34)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3808711996420,4cyclictest12-21ksoftirqd/011:30:270
3808711996419,10cyclictest3882028-21ssh11:38:300
3808711996314,42cyclictest131rcu_preempt11:05:010
380871199626,52cyclictest16950irq/224-800000009:55:020
3808711996220,10cyclictest12-21ksoftirqd/011:55:020
3808711996219,39cyclictest12-21ksoftirqd/010:07:020
3808711996219,39cyclictest12-21ksoftirqd/010:07:020
3808715996141,16cyclictest1-21systemd10:15:001
3808711996129,16cyclictest3884900-21ssh11:45:020
3808720995925,8cyclictest3848571-21kworker/2:211:31:232
3808715995812,12cyclictest23-21ksoftirqd/108:40:021
3808715995812,12cyclictest23-21ksoftirqd/108:40:021
380871199582,38cyclictest131rcu_preempt12:13:430
3808711995812,7cyclictest131rcu_preempt10:30:010
3808711995812,7cyclictest131rcu_preempt10:30:010
3808720995736,14cyclictest1-21systemd12:25:012
3808720995736,14cyclictest1-21systemd12:25:012
380871599578,5cyclictest131rcu_preempt10:55:011
380871599574,11cyclictest23-21ksoftirqd/111:40:001
3808711995713,40cyclictest12-21ksoftirqd/012:21:580
3808711995713,40cyclictest12-21ksoftirqd/012:21:580
380871599569,12cyclictest131rcu_preempt12:30:031
380871599567,11cyclictest23-21ksoftirqd/111:27:221
3808711995620,32cyclictest16850irq/134-800000011:10:010
3808711995620,32cyclictest16850irq/134-800000011:10:010
3808711995613,39cyclictest12-21ksoftirqd/010:14:550
3808711995610,6cyclictest131rcu_preempt10:40:020
3808711995610,6cyclictest131rcu_preempt10:40:020
3808711995610,6cyclictest131rcu_preempt10:00:090
3808720995546,6cyclictest3848571-21kworker/2:211:17:352
3808720995546,6cyclictest3848571-21kworker/2:211:17:352
380871599559,5cyclictest131rcu_preempt10:58:481
380871599559,5cyclictest131rcu_preempt10:58:481
380871599558,13cyclictest23-21ksoftirqd/111:20:001
380871599558,13cyclictest23-21ksoftirqd/111:20:001
380871599558,10cyclictest23-21ksoftirqd/109:00:021
380871599557,6cyclictest23-21ksoftirqd/111:49:101
380871599554,8cyclictest23-21ksoftirqd/109:10:541
380871199555,7cyclictest131rcu_preempt11:30:030
3808711995537,4cyclictest16850irq/134-800000011:25:000
380871199552,17cyclictest3864481-21unin-run11:00:010
380871199552,17cyclictest3864481-21unin-run11:00:010
3808711995519,6cyclictest16850irq/134-800000010:24:090
3808711995519,32cyclictest16850irq/134-800000010:48:200
3808711995519,32cyclictest16850irq/134-800000009:26:090
3808711995518,33cyclictest16850irq/134-800000012:07:190
380872299544,46cyclictest3898657-21ssh12:15:013
3808720995442,5cyclictest3848571-21kworker/2:211:26:352
3808720995433,15cyclictest3467602-21systemd-journal08:05:002
380871599546,10cyclictest23-21ksoftirqd/110:30:011
380871599546,10cyclictest23-21ksoftirqd/110:30:011
3808715995437,12cyclictest275-21dbus-daemon12:20:011
380871599542,13cyclictest23-21ksoftirqd/110:00:001
3808711995420,7cyclictest12-21ksoftirqd/012:35:040
3808711995420,30cyclictest16850irq/134-800000011:48:230
3808711995419,31cyclictest16850irq/134-800000012:34:510
3808711995419,31cyclictest16850irq/134-800000009:53:170
3808711995419,31cyclictest12-21ksoftirqd/012:19:180
3808711995418,6cyclictest16850irq/134-800000011:15:080
3808711995418,6cyclictest16850irq/134-800000011:15:080
3808711995418,6cyclictest16850irq/134-800000010:50:060
3808711995418,6cyclictest16850irq/134-800000010:30:040
3808711995418,32cyclictest16850irq/134-800000012:04:160
3808711995418,32cyclictest16850irq/134-800000012:04:160
3808711995418,31cyclictest16850irq/134-800000009:19:230
3808711995417,32cyclictest16850irq/134-800000009:39:520
3808711995417,32cyclictest16850irq/134-800000009:39:520
380872299535,5cyclictest131rcu_preempt10:20:013
380871599536,6cyclictest131rcu_preempt10:43:351
380871599532,11cyclictest23-21ksoftirqd/111:58:591
380871199539,40cyclictest12-21ksoftirqd/009:33:220
3808711995324,24cyclictest12-21ksoftirqd/007:45:000
3808711995318,31cyclictest16850irq/134-800000009:22:480
3808711995317,7cyclictest16850irq/134-800000009:44:550
3808711995317,32cyclictest16850irq/134-800000009:12:330
3808711995317,31cyclictest16850irq/134-800000011:10:150
3808711995311,6cyclictest131rcu_preempt09:45:070
3808722995234,13cyclictest1-21systemd07:35:003
3808720995244,5cyclictest3814129-21kworker/2:009:42:082
3808720995244,5cyclictest3813371-21kworker/2:309:50:402
380871599526,5cyclictest131rcu_preempt10:15:131
3808715995219,6cyclictest131rcu_preempt12:05:021
3808715995219,6cyclictest131rcu_preempt12:05:011
3808715995214,8cyclictest23-21ksoftirqd/110:10:011
3808715995214,8cyclictest23-21ksoftirqd/110:10:011
380871599520,17cyclictest131rcu_preempt10:05:001
3808711995218,6cyclictest16850irq/134-800000010:17:310
3808711995218,30cyclictest16850irq/134-800000011:56:090
3808711995217,7cyclictest16850irq/134-800000012:26:370
3808711995215,8cyclictest16850irq/134-800000010:43:000
3808711995215,33cyclictest12-21ksoftirqd/008:00:010
380872299511,47cyclictest0-21swapper/308:50:023
3808720995144,4cyclictest3813371-21kworker/2:309:05:022
3808720995144,4cyclictest3813371-21kworker/2:309:05:022
3808720995143,5cyclictest3848571-21kworker/2:210:55:082
3808720995143,5cyclictest3848571-21kworker/2:210:55:082
3808720995143,5cyclictest3813371-21kworker/2:310:00:222
3808720995142,6cyclictest3814129-21kworker/2:009:37:392
3808720995142,6cyclictest3814129-21kworker/2:009:37:382
3808720995142,6cyclictest3813371-21kworker/2:309:33:222
3808720995142,5cyclictest3848571-21kworker/2:211:50:482
3808720995142,5cyclictest3814129-21kworker/2:010:39:432
3808720995142,5cyclictest3814129-21kworker/2:010:39:432
380871599516,9cyclictest23-21ksoftirqd/109:36:171
380871599516,9cyclictest23-21ksoftirqd/109:36:171
380871599516,7cyclictest23-21ksoftirqd/107:10:021
3808715995135,11cyclictest275-21dbus-daemon09:55:001
3808715995131,13cyclictest3467602-21systemd-journal09:35:001
380871599512,7cyclictest23-21ksoftirqd/112:29:591
3808715995116,9cyclictest131rcu_preempt10:33:051
380871599511,5cyclictest3830608-21munin-run09:45:011
3808715995111,11cyclictest23-21ksoftirqd/111:13:251
3808720995042,5cyclictest3884915-21kworker/2:012:26:592
3808720995042,5cyclictest3848571-21kworker/2:211:04:252
3808720995041,6cyclictest3814129-21kworker/2:010:18:352
3808720995041,6cyclictest3814129-21kworker/2:009:26:312
3808720995041,6cyclictest3813371-21kworker/2:309:59:442
3808720995041,5cyclictest3848571-21kworker/2:212:35:462
3808720995041,5cyclictest3848571-21kworker/2:212:32:442
3808720995041,5cyclictest3848571-21kworker/2:212:15:292
3808720995041,5cyclictest3814129-21kworker/2:010:06:412
3808720995041,5cyclictest3814129-21kworker/2:010:06:412
3808720995041,5cyclictest3808913-21kworker/2:207:50:022
3808720995035,11cyclictest30-21ksoftirqd/209:00:022
3808720995034,11cyclictest1-21systemd07:20:002
380871599509,9cyclictest23-21ksoftirqd/109:20:011
3808715995032,13cyclictest275-21dbus-daemon10:40:011
3808715995032,13cyclictest275-21dbus-daemon10:40:011
3808722994929,15cyclictest3467602-21systemd-journal08:40:003
3808722994929,15cyclictest3467602-21systemd-journal08:40:003
3808722994922,18cyclictest37-21ksoftirqd/307:15:023
3808722994922,18cyclictest37-21ksoftirqd/307:15:023
3808720994941,5cyclictest3848571-21kworker/2:211:56:002
3808720994941,5cyclictest3848571-21kworker/2:210:33:452
3808720994941,5cyclictest3814129-21kworker/2:010:13:512
380871599492,12cyclictest23-21ksoftirqd/111:21:451
3808715994912,6cyclictest131rcu_preempt12:09:011
380871199499,5cyclictest131rcu_preempt07:39:340
3808711994926,20cyclictest3814062-21sshd08:50:330
3808722994818,21cyclictest37-21ksoftirqd/310:44:293
3808720994840,5cyclictest3855413-21kworker/2:110:45:582
3808720994839,5cyclictest3848571-21kworker/2:211:49:292
3808720994831,13cyclictest3467602-21systemd-journal10:55:022
380871599488,13cyclictest23-21ksoftirqd/112:40:011
3808715994838,7cyclictest23-21ksoftirqd/111:53:351
3808715994835,9cyclictest1-21systemd08:00:011
3808715994815,9cyclictest23-21ksoftirqd/110:22:501
380871199487,6cyclictest131rcu_preempt08:20:020
380871199481,6cyclictest131rcu_preempt09:00:000
380872299478,6cyclictest131rcu_preempt11:20:013
380872299478,6cyclictest131rcu_preempt11:20:013
380872299474,5cyclictest131rcu_preempt11:00:013
380872299474,5cyclictest131rcu_preempt11:00:003
3808722994730,13cyclictest3837336-21run-parts10:00:003
3808720994740,5cyclictest3848571-21kworker/2:210:29:322
3808720994740,5cyclictest3848571-21kworker/2:210:29:322
3808720994739,5cyclictest3848571-21kworker/2:212:05:382
3808720994739,5cyclictest3848571-21kworker/2:211:21:032
3808720994739,5cyclictest3814129-21kworker/2:009:45:302
3808720994739,5cyclictest3802181-21kworker/2:107:30:012
3808720994738,6cyclictest3848571-21kworker/2:212:12:592
3808720994738,6cyclictest3848571-21kworker/2:210:42:382
380871599475,6cyclictest131rcu_preempt10:45:421
3808715994738,6cyclictest23-21ksoftirqd/109:05:021
3808715994734,9cyclictest1-21systemd08:05:011
3808715994732,11cyclictest275-21dbus-daemon07:35:001
3808715994719,20cyclictest0-21swapper/107:40:001
3808715994716,11cyclictest23-21ksoftirqd/107:45:001
3808715994711,5cyclictest131rcu_preempt09:28:371
3808715994710,5cyclictest131rcu_preempt11:35:001
380871199479,6cyclictest131rcu_preempt09:10:000
380871199479,6cyclictest131rcu_preempt08:12:590
380871199478,5cyclictest131rcu_preempt08:40:020
380871199478,5cyclictest131rcu_preempt08:40:020
380871199471,42cyclictest3810946-21sshd07:50:330
380872299467,6cyclictest131rcu_preempt08:15:013
380872299467,5cyclictest131rcu_preempt09:00:003
380872299466,8cyclictest37-21ksoftirqd/309:10:003
3808722994636,7cyclictest37-21ksoftirqd/312:34:333
3808720994638,5cyclictest3813371-21kworker/2:310:24:592
3808720994637,6cyclictest3848571-21kworker/2:211:35:342
3808720994637,6cyclictest0-21swapper/207:10:022
3808720994637,5cyclictest3802181-21kworker/2:108:25:002
380872099463,3cyclictest131rcu_preempt09:15:452
3808715994631,10cyclictest275-21dbus-daemon09:50:011
3808715994617,7cyclictest131rcu_preempt12:15:021
3808715994614,5cyclictest131rcu_preempt12:20:211
3808715994614,5cyclictest131rcu_preempt12:20:211
3808711994636,6cyclictest19350irq/34-mmc108:25:010
3808711994631,9cyclictest12-21ksoftirqd/007:35:010
380872299459,6cyclictest131rcu_preempt10:48:573
380872299454,5cyclictest131rcu_preempt11:30:023
380872299454,32cyclictest361rcuc/307:30:013
3808720994539,3cyclictest3813371-21kworker/2:308:55:012
3808720994537,5cyclictest3848571-21kworker/2:212:01:252
3808720994537,5cyclictest3848571-21kworker/2:212:01:242
3808720994535,6cyclictest3802181-21kworker/2:107:41:172
3808720994529,12cyclictest3467602-21systemd-journal11:15:002
3808720994529,11cyclictest275-21dbus-daemon08:35:012
3808720994526,14cyclictest3467602-21systemd-journal08:45:012
380871599459,9cyclictest23-21ksoftirqd/111:44:521
380871599458,13cyclictest23-21ksoftirqd/111:05:011
380871599455,5cyclictest131rcu_preempt11:06:091
380871599455,5cyclictest131rcu_preempt11:06:091
3808715994535,6cyclictest23-21ksoftirqd/108:08:331
3808715994515,16cyclictest221rcuc/108:17:331
3808715994510,6cyclictest131rcu_preempt09:24:251
380871199454,5cyclictest131rcu_preempt07:30:010
3808722994435,7cyclictest37-21ksoftirqd/307:50:013
3808722994427,14cyclictest37-21ksoftirqd/309:49:083
380872299442,5cyclictest131rcu_preempt12:10:013
380872299442,4cyclictest131rcu_preempt11:44:073
3808722994420,17cyclictest37-21ksoftirqd/310:22:283
3808722994416,9cyclictest0-21swapper/311:55:003
3808722994412,8cyclictest3835253-21latency_hist09:55:023
3808720994436,6cyclictest3848571-21kworker/2:211:42:062
3808720994436,5cyclictest3813371-21kworker/2:309:13:472
380871599448,4cyclictest131rcu_preempt09:10:011
3808715994435,6cyclictest23-21ksoftirqd/108:25:011
3808715994432,9cyclictest23-21ksoftirqd/109:05:011
380871599442,8cyclictest23-21ksoftirqd/108:35:011
3808715994410,5cyclictest131rcu_preempt08:55:011
380871199448,5cyclictest131rcu_preempt08:03:340
3808711994425,4cyclictest131rcu_preempt07:25:000
3808711994425,4cyclictest131rcu_preempt07:25:000
3808722994329,11cyclictest37-21ksoftirqd/311:05:013
3808722994328,12cyclictest37-21ksoftirqd/312:26:553
3808722994325,13cyclictest3467602-21systemd-journal09:15:023
3808722994320,19cyclictest37-21ksoftirqd/310:37:193
3808722994320,19cyclictest37-21ksoftirqd/310:37:193
380872299431,5cyclictest131rcu_preempt11:15:013
3808720994335,5cyclictest3814129-21kworker/2:009:20:042
3808715994333,7cyclictest23-21ksoftirqd/108:45:011
3808715994330,9cyclictest23-21ksoftirqd/107:20:001
3808715994313,5cyclictest131rcu_preempt07:25:021
3808715994313,5cyclictest131rcu_preempt07:25:011
3808715994310,10cyclictest131rcu_preempt07:30:001
380871199435,6cyclictest131rcu_preempt07:20:010
3808711994327,11cyclictest3467602-21systemd-journal08:30:000
3808711994312,5cyclictest3801895-21kworker/0:008:07:190
380871199431,11cyclictest3808760-21sed07:10:020
380872299425,9cyclictest3813354-21kworker/3:009:30:013
3808722994229,9cyclictest37-21ksoftirqd/312:40:003
3808722994226,12cyclictest37-21ksoftirqd/312:18:523
380872299422,5cyclictest3809206-21munin-run07:20:013
3808722994225,12cyclictest1-21systemd07:40:013
380872299421,8cyclictest0-21swapper/309:10:383
3808722994210,10cyclictest37-21ksoftirqd/311:31:533
3808720994234,5cyclictest3855413-21kworker/2:111:07:272
3808720994234,5cyclictest3855413-21kworker/2:111:07:272
3808720994227,11cyclictest30-21ksoftirqd/207:35:012
3808711994215,23cyclictest3813909-21wc08:50:020
380872299413,7cyclictest131rcu_preempt07:25:023
380872299413,7cyclictest131rcu_preempt07:25:023
3808722994120,5cyclictest37-21ksoftirqd/309:24:453
3808722994115,9cyclictest0-21swapper/309:40:023
3808722994115,9cyclictest0-21swapper/309:40:023
380872299411,10cyclictest150-21pr/ttyS211:07:093
380872299411,10cyclictest150-21pr/ttyS211:07:083
380871599418,8cyclictest23-21ksoftirqd/107:50:021
380871599414,8cyclictest23-21ksoftirqd/108:28:181
380872299406,7cyclictest37-21ksoftirqd/308:02:593
3808722994029,8cyclictest275-21dbus-daemon08:30:003
3808722994026,10cyclictest1-21systemd08:45:003
380872299401,7cyclictest0-21swapper/310:52:003
380872299401,6cyclictest0-21swapper/311:48:123
380872299401,6cyclictest0-21swapper/309:44:023
380872299401,5cyclictest0-21swapper/310:05:423
380872299401,5cyclictest0-21swapper/310:05:423
3808722994013,7cyclictest131rcu_preempt12:00:253
3808722994013,7cyclictest131rcu_preempt12:00:253
380872299401,10cyclictest48-21kcompactd010:25:143
380872299401,10cyclictest48-21kcompactd010:25:143
3808720994023,12cyclictest275-21dbus-daemon08:10:012
380871199403,5cyclictest131rcu_preempt08:35:020
3808711994018,6cyclictest131rcu_preempt08:45:000
3808722993929,7cyclictest3826152-21latency_hist09:35:013
3808722993927,10cyclictest37-21ksoftirqd/308:20:003
3808722993926,10cyclictest3891929-21sed12:00:023
380872299391,8cyclictest0-21swapper/310:32:563
380872299391,8cyclictest0-21swapper/310:02:153
3808722993918,14cyclictest37-21ksoftirqd/311:35:533
380872299391,6cyclictest0-21swapper/311:21:143
3808720993928,6cyclictest309-21in:imuxsock08:50:012
3808715993918,17cyclictest23-21ksoftirqd/108:50:011
380872299382,4cyclictest131rcu_preempt08:35:013
380872299381,5cyclictest0-21swapper/312:24:303
380872299381,5cyclictest0-21swapper/312:24:293
380872299381,5cyclictest0-21swapper/310:12:343
3808720993829,5cyclictest3810969-21cron07:55:002
3808720993823,11cyclictest3467602-21systemd-journal07:10:022
3808720993823,11cyclictest3467602-21systemd-journal07:10:022
3808720993819,14cyclictest275-21dbus-daemon08:00:012
380871599389,5cyclictest131rcu_preempt08:15:011
380871599389,5cyclictest131rcu_preempt07:55:021
3808711993825,9cyclictest3467602-21systemd-journal07:12:330
3808711993825,9cyclictest3467602-21systemd-journal07:12:330
3808711993815,20cyclictest3810727-21latency_hist07:50:020
3808722993728,7cyclictest37-21ksoftirqd/308:24:333
380872299371,5cyclictest131rcu_preempt08:50:013
3808722993713,6cyclictest131rcu_preempt07:41:333
3808720993727,6cyclictest3802181-21kworker/2:108:17:212
3808720993722,10cyclictest275-21dbus-daemon08:40:012
3808720993722,10cyclictest275-21dbus-daemon08:40:012
3808720993711,5cyclictest131rcu_preempt07:40:012
380871599371,8cyclictest23-21ksoftirqd/107:10:331
380871599371,8cyclictest23-21ksoftirqd/107:10:331
380871199378,7cyclictest131rcu_preempt09:05:000
380871199378,7cyclictest131rcu_preempt09:05:000
3808720993616,15cyclictest3467602-21systemd-journal08:15:002
380872299352,4cyclictest131rcu_preempt09:05:023
380872299352,4cyclictest131rcu_preempt09:05:023
380872099351,8cyclictest1-21systemd09:10:002
380872099351,3cyclictest131rcu_preempt07:25:022
380872099351,3cyclictest131rcu_preempt07:25:022
3808720993426,6cyclictest30-21ksoftirqd/208:30:012
380872299335,17cyclictest131rcu_preempt08:00:033
380872299333,5cyclictest131rcu_preempt08:09:333
380872299331,4cyclictest131rcu_preempt07:52:583
3808722992918,8cyclictest3808821-21cut07:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional