You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-14 - 16:13

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot3s.osadl.org (updated Thu May 14, 2026 12:45:25)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
55061799728,33cyclictest131rcu_preempt10:40:021
550622996514,10cyclictest30-21ksoftirqd/209:45:022
550615996526,34cyclictest578434-21ssh09:55:000
550615996330,15cyclictest648367-21ssh12:20:000
550615996318,28cyclictest12-21ksoftirqd/010:53:350
55062299625,22cyclictest583679-21ssh10:05:342
550615996217,26cyclictest12-21ksoftirqd/011:05:010
550615996214,44cyclictest12-21ksoftirqd/010:13:020
55062299618,14cyclictest30-21ksoftirqd/209:53:572
55062299616,20cyclictest0-21swapper/209:39:592
550622996116,8cyclictest30-21ksoftirqd/208:25:032
550622996112,9cyclictest30-21ksoftirqd/212:40:002
550615996139,4cyclictest16850irq/134-800000010:05:020
550615996135,22cyclictest12-21ksoftirqd/010:58:030
550615996112,40cyclictest12-21ksoftirqd/007:32:430
550615996111,45cyclictest12-21ksoftirqd/008:31:430
550615996033,23cyclictest602572-21kworker/u8:310:45:000
550615996017,26cyclictest12-21ksoftirqd/009:34:240
550615996014,4cyclictest12-21ksoftirqd/010:31:480
550622995919,11cyclictest30-21ksoftirqd/210:35:452
550622995912,8cyclictest30-21ksoftirqd/211:10:012
550615995928,15cyclictest598989-21ssh10:37:280
550615995922,32cyclictest6299721ssh11:41:290
55062299588,10cyclictest30-21ksoftirqd/211:31:452
550622995815,10cyclictest30-21ksoftirqd/209:12:342
550615995826,12cyclictest12-21ksoftirqd/009:55:270
550615995826,12cyclictest12-21ksoftirqd/009:55:260
550615995825,30cyclictest12-21ksoftirqd/011:08:210
55062299579,12cyclictest30-21ksoftirqd/212:29:182
55062299573,11cyclictest30-21ksoftirqd/211:51:532
550622995723,12cyclictest131rcu_preempt11:35:352
550622995712,8cyclictest30-21ksoftirqd/212:11:202
55061599574,42cyclictest12-21ksoftirqd/009:44:260
550615995728,12cyclictest562247-21ssh09:20:590
550615995717,30cyclictest131rcu_preempt11:30:000
55062299569,8cyclictest30-21ksoftirqd/210:33:522
55062299568,10cyclictest30-21ksoftirqd/210:50:022
55062299565,9cyclictest30-21ksoftirqd/212:15:522
55062299565,11cyclictest30-21ksoftirqd/211:00:002
550622995628,9cyclictest30-21ksoftirqd/210:03:112
550622995618,9cyclictest30-21ksoftirqd/208:19:592
550622995614,7cyclictest30-21ksoftirqd/210:20:042
550615995628,13cyclictest655560-21ssh12:34:480
550615995626,14cyclictest564300-21ssh09:25:170
550615995626,14cyclictest564300-21ssh09:25:160
550615995625,13cyclictest638779-21ssh12:00:010
55062299558,7cyclictest568876-21date09:35:022
55062299557,10cyclictest30-21ksoftirqd/211:57:502
55062299556,8cyclictest575238-21ssh09:48:112
55062299556,11cyclictest30-21ksoftirqd/210:41:382
55062299555,7cyclictest30-21ksoftirqd/211:24:322
55062299555,7cyclictest30-21ksoftirqd/211:24:322
550622995519,9cyclictest30-21ksoftirqd/212:06:382
550615995545,7cyclictest16850irq/134-800000010:30:010
550615995526,14cyclictest625055-21ssh11:31:120
55062299548,7cyclictest630849-21ssh11:43:272
55062299548,15cyclictest594055-21ssh10:27:072
55062299545,12cyclictest30-21ksoftirqd/207:40:022
55062299544,11cyclictest30-21ksoftirqd/211:20:002
55062299544,11cyclictest30-21ksoftirqd/207:25:022
550622995424,21cyclictest275-21dbus-daemon09:05:012
550622995410,9cyclictest30-21ksoftirqd/209:26:572
550622995410,9cyclictest30-21ksoftirqd/209:26:572
550617995425,10cyclictest23-21ksoftirqd/110:20:011
550615995442,8cyclictest0-21swapper/009:40:000
550615995427,12cyclictest603158-21ssh10:45:560
550615995426,12cyclictest629006-21ssh11:39:470
55062299533,14cyclictest30-21ksoftirqd/210:17:322
550622995320,9cyclictest131rcu_preempt07:55:002
550622995315,9cyclictest0-21swapper/211:15:012
550622995311,5cyclictest0-21swapper/212:05:012
550617995342,8cyclictest23-21ksoftirqd/107:50:021
550617995342,8cyclictest23-21ksoftirqd/107:50:011
550617995315,22cyclictest638816-21ssh12:00:021
550615995324,13cyclictest575249-21ssh09:48:120
550615995323,13cyclictest633195-21ssh11:48:210
55062599525,4cyclictest131rcu_preempt09:15:123
55062599525,4cyclictest131rcu_preempt09:15:123
550625995212,5cyclictest131rcu_preempt12:39:433
55062299528,9cyclictest30-21ksoftirqd/211:05:012
55062299524,14cyclictest30-21ksoftirqd/211:48:082
55062299524,10cyclictest30-21ksoftirqd/209:18:152
55062299524,10cyclictest30-21ksoftirqd/209:18:142
55062299523,10cyclictest30-21ksoftirqd/208:35:002
550622995211,9cyclictest30-21ksoftirqd/209:59:052
550622995211,9cyclictest30-21ksoftirqd/209:59:052
550615995224,12cyclictest560689-21ssh09:18:090
550615995224,12cyclictest560689-21ssh09:18:090
55062299518,8cyclictest30-21ksoftirqd/207:10:022
55062299515,6cyclictest30-21ksoftirqd/210:55:022
55062299514,9cyclictest30-21ksoftirqd/211:27:432
55062299513,6cyclictest30-21ksoftirqd/209:09:592
55061599512,44cyclictest653254-21ssh12:30:000
550615995124,12cyclictest656056-21ssh12:35:210
55062299506,8cyclictest30-21ksoftirqd/209:22:112
55062299502,9cyclictest30-21ksoftirqd/212:32:522
55062299502,10cyclictest30-21ksoftirqd/212:22:402
550617995013,19cyclictest23-21ksoftirqd/112:20:031
550615995035,10cyclictest0-21swapper/008:15:010
550615995028,18cyclictest634391-21ssh11:50:310
550615995027,8cyclictest0-21swapper/011:22:350
550615995027,8cyclictest0-21swapper/011:22:350
550615995024,11cyclictest649249-21ssh12:21:320
55061599501,37cyclictest131rcu_preempt10:22:590
55061599501,30cyclictest0-21swapper/011:15:010
55062299494,9cyclictest30-21ksoftirqd/209:00:002
55062299494,5cyclictest30-21ksoftirqd/208:40:022
55062299492,11cyclictest30-21ksoftirqd/208:15:022
55061799496,22cyclictest554733-21kworker/u8:208:40:031
55061799494,3cyclictest131rcu_preempt12:01:431
550617994931,14cyclictest561548-21systemctl09:20:001
550617994931,14cyclictest561548-21systemctl09:19:591
550615994940,5cyclictest16950irq/224-800000009:10:020
550615994918,16cyclictest557496-21ssh09:11:120
55061599491,14cyclictest646124-21cat12:15:020
55062599487,4cyclictest131rcu_preempt10:38:123
55062599485,6cyclictest131rcu_preempt09:10:003
550625994829,14cyclictest3467602-21systemd-journal12:19:593
55062599482,8cyclictest556267-21munin-run09:00:003
55062299487,8cyclictest30-21ksoftirqd/207:45:032
55062299487,8cyclictest30-21ksoftirqd/207:45:032
55062299485,9cyclictest30-21ksoftirqd/208:30:022
55062299485,8cyclictest30-21ksoftirqd/208:55:012
55062299482,9cyclictest551265-21date07:20:012
550617994839,6cyclictest23-21ksoftirqd/109:54:361
55061799483,3cyclictest131rcu_preempt07:30:001
55061799483,3cyclictest131rcu_preempt07:29:591
550615994832,12cyclictest1-21systemd08:05:010
550615994829,15cyclictest275-21dbus-daemon08:50:010
550615994829,15cyclictest275-21dbus-daemon08:50:000
550615994822,10cyclictest588310-21ssh10:15:030
55062599479,5cyclictest131rcu_preempt10:52:473
55062599475,4cyclictest131rcu_preempt11:19:433
55062599475,3cyclictest131rcu_preempt11:33:473
550625994736,8cyclictest601542-21sshd10:42:433
550625994731,12cyclictest3467602-21systemd-journal08:40:003
55062299476,5cyclictest30-21ksoftirqd/207:30:442
55062299474,12cyclictest30-21ksoftirqd/210:11:072
55062299471,12cyclictest30-21ksoftirqd/207:15:022
550617994730,13cyclictest23-21ksoftirqd/109:04:591
55061799472,4cyclictest131rcu_preempt10:02:161
55061799471,4cyclictest131rcu_preempt11:13:081
550615994718,13cyclictest641641-21ssh12:05:400
55062599465,6cyclictest37-21ksoftirqd/310:11:403
55062599465,5cyclictest131rcu_preempt12:10:353
55062599465,4cyclictest131rcu_preempt11:44:023
55062599464,3cyclictest131rcu_preempt09:38:383
55062599463,4cyclictest131rcu_preempt08:45:003
55062599462,7cyclictest0-21swapper/312:25:023
55062599461,4cyclictest131rcu_preempt09:31:133
55061799465,3cyclictest131rcu_preempt12:15:001
55061799465,3cyclictest131rcu_preempt11:17:591
55061799465,3cyclictest131rcu_preempt10:35:021
550617994633,9cyclictest3467602-21systemd-journal07:40:011
55061799462,3cyclictest131rcu_preempt07:20:001
55061799461,3cyclictest131rcu_preempt10:00:031
55061799461,3cyclictest131rcu_preempt10:00:021
550615994627,15cyclictest640434-21ssh12:03:190
55061599461,41cyclictest583967-21ssh10:06:080
55062599455,3cyclictest131rcu_preempt10:01:243
55062599454,9cyclictest588112-21kworker/3:111:51:043
55062599454,3cyclictest131rcu_preempt09:56:463
55062599454,3cyclictest131rcu_preempt09:56:463
550625994527,14cyclictest639737-21sshd12:01:433
550625994522,5cyclictest37-21ksoftirqd/310:33:343
550625994521,5cyclictest37-21ksoftirqd/311:28:193
55062299456,7cyclictest30-21ksoftirqd/208:10:022
55061799454,9cyclictest311-21rs:main1
55061799454,3cyclictest131rcu_preempt09:22:201
55061799454,3cyclictest131rcu_preempt08:20:001
550617994520,9cyclictest23-21ksoftirqd/110:48:351
55061799451,4cyclictest131rcu_preempt11:30:301
55061799451,3cyclictest131rcu_preempt12:10:011
55061599452,8cyclictest12-21ksoftirqd/008:40:010
550615994526,4cyclictest0-21swapper/011:15:470
550615994521,16cyclictest275-21dbus-daemon07:45:000
55062599448,6cyclictest131rcu_preempt09:26:513
55062599448,6cyclictest131rcu_preempt09:26:513
55062599444,4cyclictest131rcu_preempt10:25:113
550625994433,6cyclictest554189-21sendmail08:20:003
550625994423,5cyclictest37-21ksoftirqd/308:30:013
55062599441,3cyclictest131rcu_preempt10:21:143
550625994411,17cyclictest556789-21diskmemload10:20:033
55062299446,5cyclictest30-21ksoftirqd/208:50:032
55062299446,5cyclictest30-21ksoftirqd/208:50:022
55062299441,12cyclictest553449-21sed08:05:012
55061799444,3cyclictest131rcu_preempt12:25:241
55061799444,3cyclictest131rcu_preempt11:41:411
55061799444,3cyclictest131rcu_preempt11:09:031
55061799443,4cyclictest131rcu_preempt10:10:121
55061799443,3cyclictest131rcu_preempt11:46:591
55061799443,3cyclictest131rcu_preempt10:20:361
55061799443,3cyclictest131rcu_preempt08:55:011
550617994426,13cyclictest3467602-21systemd-journal11:00:011
55061799442,4cyclictest131rcu_preempt09:30:091
55061799442,3cyclictest131rcu_preempt12:33:201
55061799442,3cyclictest131rcu_preempt09:25:521
55061799442,3cyclictest131rcu_preempt09:25:521
550617994423,15cyclictest275-21dbus-daemon11:55:011
55061799441,3cyclictest131rcu_preempt10:55:011
55061799441,3cyclictest131rcu_preempt09:40:011
550615994437,4cyclictest16950irq/224-800000008:30:000
550615994433,7cyclictest552097-21modprobe07:40:010
55062599438,6cyclictest131rcu_preempt11:06:033
55062599436,6cyclictest131rcu_preempt09:50:013
55062599436,5cyclictest131rcu_preempt09:50:273
55062599433,6cyclictest0-21swapper/309:42:583
55062599433,3cyclictest131rcu_preempt08:05:033
550625994331,9cyclictest651336-21sshd12:25:433
550625994317,5cyclictest37-21ksoftirqd/310:08:563
55062599431,3cyclictest131rcu_preempt12:30:593
55062599431,3cyclictest131rcu_preempt11:45:223
55062599431,3cyclictest131rcu_preempt11:00:423
55062299435,10cyclictest30-21ksoftirqd/207:45:022
55062299433,8cyclictest30-21ksoftirqd/207:30:012
55062299433,8cyclictest30-21ksoftirqd/207:30:002
55062299431,10cyclictest30-21ksoftirqd/208:45:022
55061799439,6cyclictest131rcu_preempt11:27:441
55061799434,4cyclictest131rcu_preempt08:44:591
55061799434,3cyclictest131rcu_preempt11:25:011
55061799434,3cyclictest131rcu_preempt11:25:011
55061799434,3cyclictest131rcu_preempt09:11:511
55061799434,3cyclictest131rcu_preempt08:05:021
55061799433,3cyclictest131rcu_preempt12:37:571
55061799433,3cyclictest131rcu_preempt09:41:451
550617994330,10cyclictest3467602-21systemd-journal08:10:011
55061799432,3cyclictest131rcu_preempt08:56:431
55061799431,3cyclictest131rcu_preempt10:25:041
55061799431,3cyclictest131rcu_preempt09:10:001
55061799431,3cyclictest131rcu_preempt07:59:591
550615994329,11cyclictest3467602-21systemd-journal07:30:000
550615994329,11cyclictest3467602-21systemd-journal07:29:590
55061599432,11cyclictest554434-21sshd08:20:430
550615994319,18cyclictest275-21dbus-daemon08:45:010
55061599431,37cyclictest151rcuc/008:00:000
55062599426,8cyclictest37-21ksoftirqd/311:58:283
55062599425,4cyclictest131rcu_preempt11:36:313
55062599424,4cyclictest131rcu_preempt07:40:013
550625994225,11cyclictest37-21ksoftirqd/307:30:003
550625994225,11cyclictest37-21ksoftirqd/307:30:003
55061799422,4cyclictest131rcu_preempt11:37:111
55061799421,3cyclictest131rcu_preempt12:20:011
55061799421,3cyclictest131rcu_preempt10:06:571
55061799421,3cyclictest131rcu_preempt07:25:031
550615994234,5cyclictest16950irq/224-800000008:20:000
550615994228,10cyclictest1-21systemd08:55:010
55062599415,7cyclictest37-21ksoftirqd/310:56:293
55062599412,7cyclictest551868-21latency_hist07:35:013
550625994118,5cyclictest37-21ksoftirqd/307:47:443
550625994118,5cyclictest37-21ksoftirqd/307:47:433
550625994118,4cyclictest37-21ksoftirqd/310:47:373
55062299412,8cyclictest30-21ksoftirqd/208:00:022
55061799413,3cyclictest131rcu_preempt10:40:421
55061799412,3cyclictest131rcu_preempt09:45:061
55061799412,3cyclictest131rcu_preempt07:50:431
55061799411,3cyclictest131rcu_preempt08:50:041
55061799411,3cyclictest131rcu_preempt08:50:031
55061799411,3cyclictest131rcu_preempt08:33:431
550615994129,6cyclictest309-21in:imuxsock08:10:000
55062599405,4cyclictest131rcu_preempt08:25:013
55062599404,6cyclictest37-21ksoftirqd/309:22:283
550625994018,4cyclictest37-21ksoftirqd/307:15:033
55062599401,7cyclictest0-21swapper/311:10:213
55061799401,3cyclictest131rcu_preempt08:28:421
55061799401,3cyclictest131rcu_preempt07:15:011
550615994033,4cyclictest16950irq/224-800000007:21:390
550615994021,13cyclictest3467602-21systemd-journal08:59:590
55061599401,2cyclictest0-21swapper/007:15:030
55062599397,8cyclictest37-21ksoftirqd/311:23:193
55062599397,8cyclictest37-21ksoftirqd/311:23:193
55062599394,3cyclictest131rcu_preempt12:06:303
55062599391,11cyclictest131rcu_preempt09:12:523
55061599391,7cyclictest0-21swapper/007:10:020
55061599391,34cyclictest551104-21systemd-run07:20:000
55062599385,3cyclictest131rcu_preempt08:52:433
55061799381,3cyclictest131rcu_preempt07:45:021
55061799381,3cyclictest131rcu_preempt07:10:021
550625993727,7cyclictest37-21ksoftirqd/308:50:023
550625993727,7cyclictest37-21ksoftirqd/308:50:013
550625993720,13cyclictest3467602-21systemd-journal07:25:003
550617993726,8cyclictest1-21systemd08:25:011
55061799371,3cyclictest131rcu_preempt08:13:421
55061599372,6cyclictest3467602-21systemd-journal09:03:110
55062599364,3cyclictest131rcu_preempt08:13:433
55062599363,4cyclictest131rcu_preempt09:04:593
55061599361,32cyclictest0-21swapper/007:50:570
55062599354,3cyclictest131rcu_preempt08:05:033
550625993524,8cyclictest37-21ksoftirqd/308:35:013
55061799351,3cyclictest131rcu_preempt07:35:001
550625993422,8cyclictest3467602-21systemd-journal07:51:433
55061599341,4cyclictest0-21swapper/007:47:190
55061599341,4cyclictest0-21swapper/007:47:190
550625993325,5cyclictest551067-21sendmail07:20:003
55062599315,3cyclictest131rcu_preempt08:00:013
550625992922,4cyclictest3467562-21sshd07:43:433
55062599274,3cyclictest131rcu_preempt07:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional