You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-03 - 18:10

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot3s.osadl.org (updated Wed Jun 03, 2026 12:45:27)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2859521998870,15cyclictest30-21ksoftirqd/207:25:022
2859512997459,11cyclictest2915122-21run-parts11:00:000
2859512997459,11cyclictest2915122-21run-parts11:00:000
285951299741,11cyclictest121ksoftirqd/011:40:000
285951299681,42cyclictest0-21swapper/010:11:320
2859512996711,39cyclictest16950irq/224-800000010:25:010
2859512996525,10cyclictest2876922-21ssh09:35:020
285952399649,13cyclictest37-21ksoftirqd/311:50:013
2859521996438,7cyclictest2876907-21kworker/2:210:47:142
2859523996318,9cyclictest131rcu_preempt11:54:453
2859523996222,9cyclictest131rcu_preempt09:12:013
2859512996224,33cyclictest2939939-21latency_hist11:55:020
2859523996119,11cyclictest37-21ksoftirqd/311:09:223
285951299618,34cyclictest2859515-21latency_hist07:10:020
285951299618,34cyclictest2859515-21latency_hist07:10:010
2859512996128,29cyclictest12-21ksoftirqd/012:11:020
2859512996117,40cyclictest12-21ksoftirqd/010:44:170
2859512996111,45cyclictest131rcu_preempt11:42:430
2859523996022,9cyclictest131rcu_preempt07:45:033
285951799601,38cyclictest2861305-21kworker/u8:308:40:011
285951799601,29cyclictest2910882-21kworker/u8:110:55:021
285951799601,29cyclictest2910882-21kworker/u8:110:55:011
285951299606,7cyclictest12-21ksoftirqd/009:15:010
285952399595,16cyclictest2909175-21ssh10:46:323
2859523995918,10cyclictest131rcu_preempt10:40:153
2859521995944,5cyclictest2852098-21kworker/2:107:20:002
285951299598,34cyclictest151rcuc/011:30:030
285952399585,8cyclictest37-21ksoftirqd/312:32:103
2859523995814,11cyclictest0-21swapper/312:01:153
2859523995812,10cyclictest37-21ksoftirqd/311:30:013
2859517995851,4cyclictest23-21ksoftirqd/109:55:011
2859512995832,5cyclictest131rcu_preempt10:06:290
2859512995811,6cyclictest131rcu_preempt09:45:010
2859523995716,9cyclictest37-21ksoftirqd/310:35:353
2859523995716,9cyclictest37-21ksoftirqd/310:35:353
2859523995710,9cyclictest37-21ksoftirqd/310:25:483
2859523995710,9cyclictest37-21ksoftirqd/310:25:483
2859523995710,8cyclictest37-21ksoftirqd/311:25:013
2859521995748,6cyclictest2865495-21kworker/2:309:39:312
2859521995748,6cyclictest2865495-21kworker/2:309:39:312
285952399568,12cyclictest37-21ksoftirqd/312:37:443
285952399566,7cyclictest37-21ksoftirqd/310:17:113
285952399565,13cyclictest37-21ksoftirqd/311:20:013
2859523995615,7cyclictest37-21ksoftirqd/309:46:283
2859523995615,7cyclictest37-21ksoftirqd/309:46:273
2859523995614,7cyclictest37-21ksoftirqd/311:40:133
2859523995614,7cyclictest37-21ksoftirqd/311:40:023
285951299569,13cyclictest12-21ksoftirqd/009:40:010
285951299569,13cyclictest12-21ksoftirqd/009:40:000
2859512995617,34cyclictest309-21in:imuxsock09:10:020
285952399557,11cyclictest37-21ksoftirqd/307:50:023
285952399554,11cyclictest37-21ksoftirqd/312:19:023
2859523995514,8cyclictest37-21ksoftirqd/309:20:013
2859523995513,9cyclictest37-21ksoftirqd/311:14:243
2859523995511,8cyclictest37-21ksoftirqd/310:53:573
2859512995548,5cyclictest12-21ksoftirqd/009:02:300
285952399549,9cyclictest37-21ksoftirqd/311:59:363
285952399547,9cyclictest37-21ksoftirqd/312:05:563
285952399546,11cyclictest37-21ksoftirqd/310:05:033
285952399545,9cyclictest37-21ksoftirqd/310:00:023
285952399544,10cyclictest37-21ksoftirqd/309:43:113
2859523995431,18cyclictest275-21dbus-daemon09:10:013
2859523995415,7cyclictest37-21ksoftirqd/310:25:023
2859523995411,8cyclictest0-21swapper/310:12:593
2859512995424,13cyclictest2918149-21ssh11:06:170
2859512995419,10cyclictest12-21ksoftirqd/011:14:100
2859512995416,34cyclictest12-21ksoftirqd/009:45:320
2859512995416,34cyclictest12-21ksoftirqd/009:45:310
2859512995415,21cyclictest16850irq/134-800000012:00:030
285951299541,42cyclictest12-21ksoftirqd/009:55:580
285951299541,38cyclictest2871847-21ssh09:23:330
285952399538,9cyclictest37-21ksoftirqd/308:40:013
285952399537,10cyclictest37-21ksoftirqd/312:20:283
285952399536,12cyclictest37-21ksoftirqd/311:35:023
285952399535,9cyclictest37-21ksoftirqd/309:29:083
2859523995317,10cyclictest37-21ksoftirqd/309:20:243
2859523995311,14cyclictest37-21ksoftirqd/312:10:253
2859521995344,5cyclictest2876907-21kworker/2:211:16:432
2859521995343,6cyclictest2859735-21kworker/2:208:16:512
285951299532,29cyclictest3467602-21systemd-journal09:29:200
2859512995315,34cyclictest151rcuc/010:05:010
2859512995314,34cyclictest16850irq/134-800000011:34:450
285951299531,16cyclictest2955543-21latency_hist12:30:020
285952399528,9cyclictest2875488-21ssh09:31:343
2859521995235,12cyclictest275-21dbus-daemon08:35:012
2859521995235,12cyclictest275-21dbus-daemon08:35:002
2859512995240,9cyclictest12-21ksoftirqd/010:46:280
2859512995224,13cyclictest2950771-21ssh12:19:270
2859512995215,10cyclictest12-21ksoftirqd/010:50:160
2859512995214,12cyclictest12-21ksoftirqd/012:10:020
2859512995212,28cyclictest131rcu_preempt12:37:520
285952399519,9cyclictest37-21ksoftirqd/309:38:153
285952399519,9cyclictest37-21ksoftirqd/309:38:153
285952399515,9cyclictest37-21ksoftirqd/310:31:113
285952399514,8cyclictest37-21ksoftirqd/308:05:013
2859523995143,4cyclictest0-21swapper/307:10:023
2859523995143,4cyclictest0-21swapper/307:10:013
2859521995143,5cyclictest2953254-21kworker/2:212:36:592
2859521995143,5cyclictest2876907-21kworker/2:210:04:322
2859521995143,5cyclictest2876907-21kworker/2:209:58:192
2859521995143,5cyclictest2865495-21kworker/2:310:44:182
2859521995142,6cyclictest2865495-21kworker/2:312:13:492
2859521995141,5cyclictest2865495-21kworker/2:311:40:072
2859521995136,11cyclictest1-21systemd09:25:012
2859517995142,5cyclictest2859519-21ls07:10:021
2859517995142,5cyclictest2859519-21ls07:10:011
285951299515,28cyclictest2906258-21ssh10:40:030
285951299515,28cyclictest2906258-21ssh10:40:020
2859512995144,4cyclictest16950irq/224-800000008:00:000
285952399508,13cyclictest37-21ksoftirqd/311:00:233
2859523995016,8cyclictest37-21ksoftirqd/310:08:133
2859521995043,5cyclictest2876907-21kworker/2:211:27:082
2859521995042,5cyclictest2876907-21kworker/2:211:20:112
2859521995042,5cyclictest2865495-21kworker/2:312:34:012
2859521995042,5cyclictest2865495-21kworker/2:311:47:062
2859521995042,5cyclictest2865495-21kworker/2:310:36:492
2859521995042,5cyclictest2865495-21kworker/2:310:36:482
2859521995041,5cyclictest2865495-21kworker/2:311:33:392
2859517995031,16cyclictest23-21ksoftirqd/111:25:111
285951299507,11cyclictest12-21ksoftirqd/008:35:020
285951299507,11cyclictest12-21ksoftirqd/008:35:010
285951299502,7cyclictest2864452-21systemd-run08:45:010
285952399496,14cyclictest37-21ksoftirqd/310:59:563
285952399496,14cyclictest37-21ksoftirqd/310:59:553
285952399496,11cyclictest37-21ksoftirqd/308:55:013
285952399494,9cyclictest37-21ksoftirqd/308:45:023
285952399494,10cyclictest37-21ksoftirqd/307:30:033
2859523994932,12cyclictest3467602-21systemd-journal09:50:293
2859521994941,5cyclictest2865495-21kworker/2:312:22:152
2859521994941,5cyclictest2865495-21kworker/2:312:06:442
2859521994941,5cyclictest2865495-21kworker/2:311:37:192
2859521994941,5cyclictest2865495-21kworker/2:311:11:362
2859521994941,5cyclictest2865495-21kworker/2:309:16:082
2859521994941,5cyclictest2864722-21kworker/2:108:58:372
2859521994941,5cyclictest2864722-21kworker/2:108:58:362
2859521994940,6cyclictest2865495-21kworker/2:312:02:032
2859521994940,5cyclictest2865495-21kworker/2:309:14:192
2859521994934,11cyclictest2885629-21sshd09:54:292
2859521994931,13cyclictest30-21ksoftirqd/212:26:422
2859512994940,6cyclictest12-21ksoftirqd/011:49:480
2859512994932,13cyclictest1-21systemd07:30:000
285952399487,7cyclictest37-21ksoftirqd/309:05:013
285952399486,8cyclictest37-21ksoftirqd/309:00:013
285952399486,8cyclictest37-21ksoftirqd/309:00:003
285952399486,6cyclictest2862618-21latency_hist08:10:023
285952399481,9cyclictest2861163-21sed07:40:023
2859523994812,6cyclictest0-21swapper/307:14:003
285952399481,14cyclictest2862996-21sed08:15:013
2859521994841,5cyclictest2944288-21kworker/2:112:17:522
2859521994840,5cyclictest2876907-21kworker/2:210:13:052
2859521994839,6cyclictest2862396-21kworker/2:008:30:022
2859521994830,14cyclictest2865495-21kworker/2:310:33:112
2859517994833,12cyclictest2908383-21systemctl10:45:001
285951299485,10cyclictest12-21ksoftirqd/011:16:000
285951299483,7cyclictest12-21ksoftirqd/008:25:050
285951299483,7cyclictest121ksoftirqd/009:18:160
2859512994836,8cyclictest12-21ksoftirqd/009:50:050
2859512994832,13cyclictest12-21ksoftirqd/012:20:200
2859512994831,13cyclictest3467602-21systemd-journal07:14:210
2859512994826,17cyclictest12-21ksoftirqd/012:30:480
285951299481,32cyclictest12-21ksoftirqd/012:03:410
285951299481,14cyclictest2915528-21ssh11:00:260
285951299481,13cyclictest12-21ksoftirqd/007:25:010
2859512994810,8cyclictest12-21ksoftirqd/010:30:040
2859523994730,13cyclictest3467602-21systemd-journal07:59:303
2859523994711,7cyclictest37-21ksoftirqd/312:25:423
2859521994739,5cyclictest2876907-21kworker/2:210:08:402
2859521994739,5cyclictest2865495-21kworker/2:311:00:592
2859521994739,5cyclictest2865495-21kworker/2:310:55:592
2859521994739,5cyclictest2865495-21kworker/2:310:55:582
2859521994731,11cyclictest30-21ksoftirqd/208:15:012
285951799476,6cyclictest0-21swapper/111:17:141
2859512994732,12cyclictest12-21ksoftirqd/010:27:300
2859512994732,12cyclictest12-21ksoftirqd/010:27:300
2859512994715,28cyclictest309-21in:imuxsock10:17:300
285951299471,12cyclictest12-21ksoftirqd/008:20:030
285952399469,9cyclictest0-21swapper/308:50:013
285952399467,8cyclictest37-21ksoftirqd/308:30:003
285952399466,8cyclictest37-21ksoftirqd/308:20:003
285952399466,6cyclictest2860812-21systemd-run07:35:003
2859521994639,5cyclictest2865495-21kworker/2:309:25:342
2859521994637,5cyclictest2865495-21kworker/2:310:52:222
2859521994612,12cyclictest131rcu_preempt09:41:172
285951799468,5cyclictest131rcu_preempt10:35:001
2859517994633,9cyclictest23-21ksoftirqd/112:35:011
2859517994629,12cyclictest309-21in:imuxsock09:20:001
2859517994628,13cyclictest1-21systemd07:40:001
2859512994615,27cyclictest16850irq/134-800000011:22:440
2859521994537,5cyclictest2876907-21kworker/2:211:59:232
2859521994537,5cyclictest2865495-21kworker/2:311:53:372
2859517994534,8cyclictest2872438-21cron09:25:011
2859517994533,7cyclictest2861333-21grep07:45:001
2859517994524,16cyclictest1-21systemd07:26:301
2859517994523,16cyclictest275-21dbus-daemon10:10:011
285951799452,10cyclictest2935279-21ssh11:44:531
285951299454,10cyclictest12-21ksoftirqd/008:15:000
2859512994534,8cyclictest151rcuc/007:40:000
2859521994436,5cyclictest2859735-21kworker/2:207:51:152
2859521994431,9cyclictest30-21ksoftirqd/210:27:012
2859521994431,9cyclictest30-21ksoftirqd/210:27:002
2859517994426,4cyclictest23-21ksoftirqd/111:08:551
285952399436,7cyclictest37-21ksoftirqd/307:20:003
2859521994328,11cyclictest30-21ksoftirqd/209:49:132
2859521994328,11cyclictest30-21ksoftirqd/209:49:132
2859521994327,12cyclictest275-21dbus-daemon09:35:002
2859517994330,10cyclictest275-21dbus-daemon12:10:011
2859517994330,10cyclictest23-21ksoftirqd/109:15:021
2859517994325,13cyclictest275-21dbus-daemon11:35:011
285951799432,3cyclictest131rcu_preempt10:17:501
285951299432,9cyclictest12-21ksoftirqd/009:00:010
285951299432,9cyclictest12-21ksoftirqd/009:00:000
285951299432,12cyclictest12-21ksoftirqd/008:29:590
285952399429,8cyclictest0-21swapper/307:55:013
285952399426,8cyclictest37-21ksoftirqd/308:35:013
285952399426,8cyclictest37-21ksoftirqd/308:35:003
285952199429,20cyclictest2859505-21cyclictest07:35:022
2859521994234,5cyclictest2876907-21kworker/2:210:15:502
2859521994231,9cyclictest30-21ksoftirqd/207:15:012
2859521994230,9cyclictest30-21ksoftirqd/211:06:052
285951799423,3cyclictest131rcu_preempt12:18:521
2859517994232,6cyclictest2865733-21diskmemload11:20:031
2859517994226,11cyclictest3467602-21systemd-journal08:55:001
2859517994225,13cyclictest275-21dbus-daemon10:50:011
2859517994219,13cyclictest23-21ksoftirqd/111:49:301
2859517994210,7cyclictest0-21swapper/110:01:221
285951299422,5cyclictest2859993-21grep07:20:000
2859521994134,5cyclictest2865495-21kworker/2:310:23:112
2859521994129,8cyclictest1-21systemd09:05:012
2859521994127,11cyclictest30-21ksoftirqd/208:40:022
285951799413,3cyclictest131rcu_preempt11:54:291
2859517994131,7cyclictest2940260-21sshd11:55:291
2859517994128,10cyclictest0-21swapper/108:35:011
2859517994128,10cyclictest0-21swapper/108:35:001
2859517994124,14cyclictest2865221-21tr09:00:011
2859517994124,14cyclictest2865221-21tr09:00:011
2859517994123,14cyclictest1-21systemd08:15:011
2859512994134,4cyclictest16950irq/224-800000008:39:590
2859521994018,18cyclictest30-21ksoftirqd/207:35:132
285951799405,4cyclictest131rcu_preempt11:11:431
285951799403,3cyclictest131rcu_preempt09:46:271
285951799403,3cyclictest131rcu_preempt09:46:261
2859517994027,10cyclictest23-21ksoftirqd/112:24:381
2859517994027,10cyclictest23-21ksoftirqd/107:55:011
2859517994026,11cyclictest2881456-21cat09:45:021
2859517994025,11cyclictest275-21dbus-daemon12:30:011
2859517994024,12cyclictest3467602-21systemd-journal12:05:021
285951799401,9cyclictest2958217-21ssh12:35:381
2859512994033,5cyclictest12-21ksoftirqd/007:48:300
285951299401,35cyclictest2860768-21sshd07:31:280
285952399398,5cyclictest131rcu_preempt08:24:303
2859517993929,7cyclictest2933085-21cron11:40:011
2859517993928,8cyclictest23-21ksoftirqd/111:00:431
2859517993924,11cyclictest275-21dbus-daemon12:15:011
2859517993923,12cyclictest275-21dbus-daemon08:45:001
285951799391,3cyclictest131rcu_preempt09:28:131
285951299398,5cyclictest131rcu_preempt08:50:000
285951299397,5cyclictest131rcu_preempt08:55:020
2859512993929,7cyclictest12-21ksoftirqd/008:10:020
2859523993815,16cyclictest2860318-21cut07:25:023
2859521993820,13cyclictest275-21dbus-daemon08:00:002
285951799384,3cyclictest131rcu_preempt09:38:321
285951799384,3cyclictest131rcu_preempt09:38:311
2859517993823,12cyclictest23-21ksoftirqd/109:32:301
285951299384,11cyclictest12-21ksoftirqd/007:43:380
285952199375,3cyclictest131rcu_preempt09:09:592
2859521993729,5cyclictest2862396-21kworker/2:008:47:282
2859521993726,8cyclictest30-21ksoftirqd/207:41:062
2859517993730,4cyclictest2862357-21cron08:05:001
2859517993726,8cyclictest23-21ksoftirqd/107:59:301
2859517993725,9cyclictest2860430-21sort07:25:011
285951799372,3cyclictest131rcu_preempt09:05:001
2859517993723,11cyclictest23-21ksoftirqd/110:28:031
2859517993723,11cyclictest23-21ksoftirqd/110:28:021
2859517993715,6cyclictest23-21ksoftirqd/110:37:281
2859517993715,6cyclictest23-21ksoftirqd/110:37:271
2859521993630,4cyclictest2859735-21kworker/2:207:26:312
2859517993629,4cyclictest1-21systemd08:10:011
2859517993626,8cyclictest23-21ksoftirqd/110:12:371
2859517993622,11cyclictest23-21ksoftirqd/110:55:001
2859517993621,11cyclictest23-21ksoftirqd/109:50:371
2859512993628,6cyclictest12-21ksoftirqd/008:05:020
285951799353,3cyclictest131rcu_preempt10:25:021
285951799353,3cyclictest131rcu_preempt08:45:291
2859517993525,6cyclictest2859931-21cron07:17:011
2859517993523,9cyclictest2863635-21sshd08:27:301
2859521993421,9cyclictest30-21ksoftirqd/208:45:012
2859521993419,12cyclictest1-21systemd08:25:002
285951799341,3cyclictest131rcu_preempt08:20:001
2859512993422,8cyclictest12-21ksoftirqd/007:54:320
2859521993324,6cyclictest271irq_work/208:05:032
2859521993322,8cyclictest30-21ksoftirqd/208:51:302
285951799333,21cyclictest2865739-21munin-run09:10:001
2859517993323,5cyclictest309-21in:imuxsock07:50:001
2859517993319,11cyclictest2859691-21sshd07:14:201
285952199326,3cyclictest131rcu_preempt08:10:002
285952199307,6cyclictest131rcu_preempt07:50:032
2859521993020,7cyclictest2859529-21date07:10:022
2859521993020,7cyclictest2859529-21date07:10:012
2859517993018,9cyclictest3467602-21systemd-journal08:23:291
285951799281,5cyclictest0-21swapper/107:31:281
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional