You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-16 - 16:01

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot3s.osadl.org (updated Thu Apr 16, 2026 12:45:26)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
808209912310,92cyclictest108055-21ssh10:00:030
8082099114100,6cyclictest0-21swapper/011:15:020
808289911398,11cyclictest30-21ksoftirqd/210:55:022
808289911398,11cyclictest30-21ksoftirqd/210:55:022
80820991071,94cyclictest150-21pr/ttyS212:15:030
80828997846,6cyclictest97477-21kworker/2:310:50:012
80820997827,45cyclictest3467602-21systemd-journal11:40:010
80828996713,12cyclictest30-21ksoftirqd/209:31:572
8082899667,18cyclictest91511-21ssh09:20:132
80828996612,9cyclictest30-21ksoftirqd/210:14:212
80828996514,13cyclictest30-21ksoftirqd/212:00:332
80828996417,10cyclictest30-21ksoftirqd/211:50:032
80828996417,10cyclictest30-21ksoftirqd/211:50:032
80828996315,10cyclictest30-21ksoftirqd/210:17:542
8082899627,15cyclictest119111-21ssh10:26:112
80828996215,9cyclictest30-21ksoftirqd/210:34:192
80828996215,12cyclictest30-21ksoftirqd/211:24:192
8082899612,20cyclictest105219-21ssh09:53:232
80828996115,9cyclictest30-21ksoftirqd/211:11:032
80828996114,10cyclictest30-21ksoftirqd/212:17:572
80826996122,7cyclictest23-21ksoftirqd/109:15:031
80820996120,23cyclictest157165-21ssh11:57:140
80828996014,10cyclictest30-21ksoftirqd/209:36:482
80820996028,29cyclictest12-21ksoftirqd/009:25:160
80828995915,8cyclictest30-21ksoftirqd/209:27:122
80828995914,9cyclictest30-21ksoftirqd/212:07:542
80828995914,11cyclictest30-21ksoftirqd/209:42:062
80828995914,11cyclictest30-21ksoftirqd/209:42:062
80828995913,11cyclictest30-21ksoftirqd/211:56:482
80828995912,9cyclictest168558-21ssh12:24:422
80828995910,13cyclictest134614-21ssh11:03:272
80828995910,13cyclictest134614-21ssh11:03:272
8082899588,7cyclictest30-21ksoftirqd/210:05:462
80828995810,12cyclictest150189-21rm11:40:182
80826995817,12cyclictest23-21ksoftirqd/110:05:011
80820995829,13cyclictest124256-21ssh10:38:430
80820995828,13cyclictest91864-21ssh09:21:070
8082099581,50cyclictest141462-21sendmail11:20:000
8082899579,12cyclictest30-21ksoftirqd/212:25:372
80828995716,8cyclictest30-21ksoftirqd/211:37:572
80828995712,9cyclictest30-21ksoftirqd/212:31:362
80828995711,10cyclictest30-21ksoftirqd/212:38:272
80828995711,10cyclictest30-21ksoftirqd/207:30:012
80828995710,9cyclictest30-21ksoftirqd/211:32:122
80828995710,10cyclictest30-21ksoftirqd/210:41:062
80828995710,10cyclictest30-21ksoftirqd/210:41:062
80820995723,11cyclictest12-21ksoftirqd/011:33:530
80828995613,7cyclictest30-21ksoftirqd/211:50:392
80828995610,9cyclictest30-21ksoftirqd/211:28:202
80828995610,9cyclictest30-21ksoftirqd/210:39:302
8082699566,34cyclictest23-21ksoftirqd/111:50:021
8082699566,34cyclictest23-21ksoftirqd/111:50:021
80826995616,10cyclictest23-21ksoftirqd/108:00:011
80826995616,10cyclictest23-21ksoftirqd/108:00:011
80820995646,5cyclictest86991-21diskmemload12:35:000
80820995627,14cyclictest155663-21ssh11:53:430
8082899556,15cyclictest116860-21ssh10:20:432
8082899554,14cyclictest102079-21ssh09:45:342
80828995510,10cyclictest30-21ksoftirqd/211:19:182
8082899550,21cyclictest82378-21cut07:40:012
80831995437,13cyclictest1-21systemd09:55:013
8082899549,7cyclictest30-21ksoftirqd/209:11:592
8082899545,4cyclictest131rcu_preempt10:05:022
80828995410,10cyclictest30-21ksoftirqd/210:55:252
80826995439,11cyclictest1-21systemd09:45:011
80826995439,11cyclictest1-21systemd09:45:001
80826995413,9cyclictest131rcu_preempt10:30:321
80820995430,7cyclictest12-21ksoftirqd/010:15:020
80820995417,34cyclictest12-21ksoftirqd/010:16:540
80820995410,25cyclictest19350irq/34-mmc111:05:020
80820995410,25cyclictest19350irq/34-mmc111:05:020
80828995312,10cyclictest30-21ksoftirqd/207:55:012
80828995310,8cyclictest30-21ksoftirqd/209:19:062
8082699539,13cyclictest23-21ksoftirqd/111:12:571
80820995313,10cyclictest1-21systemd10:10:010
80826995238,10cyclictest1-21systemd09:50:001
80826995213,8cyclictest131rcu_preempt11:56:221
80820995228,3cyclictest131rcu_preempt09:33:240
80820995228,21cyclictest137219-21ssh11:09:470
80820995223,3cyclictest0-21swapper/009:11:550
80820995218,31cyclictest12-21ksoftirqd/010:00:150
8082899519,8cyclictest364-21snmpd08:47:482
80828995111,9cyclictest30-21ksoftirqd/207:20:012
80826995112,22cyclictest99742-21ssh09:40:021
80820995141,6cyclictest16850irq/134-800000009:45:010
80820995141,6cyclictest16850irq/134-800000009:45:010
80820995135,11cyclictest91162-21cron09:20:000
80820995129,16cyclictest160318-21ssh12:05:010
80820995128,19cyclictest99162-21ssh09:38:590
8082099512,12cyclictest118462-21cron10:25:000
80820995121,14cyclictest165437-21ssh12:16:590
8082099511,44cyclictest168416-21ssh12:24:140
8082899507,8cyclictest30-21ksoftirqd/212:11:362
80828995010,9cyclictest30-21ksoftirqd/211:06:342
8082699507,13cyclictest23-21ksoftirqd/110:53:041
8082699507,13cyclictest23-21ksoftirqd/110:53:041
80826995038,8cyclictest23-21ksoftirqd/109:10:021
80826995016,11cyclictest23-21ksoftirqd/112:15:021
80826995013,9cyclictest131rcu_preempt10:55:581
8082099502,29cyclictest0-21swapper/010:52:380
8082099502,29cyclictest0-21swapper/010:52:380
8082899499,9cyclictest30-21ksoftirqd/209:56:562
80828994928,15cyclictest275-21dbus-daemon07:50:012
8082899492,18cyclictest83576-21latency_hist08:00:032
8082699496,12cyclictest23-21ksoftirqd/112:34:111
8082699495,5cyclictest131rcu_preempt07:55:011
80826994917,8cyclictest23-21ksoftirqd/112:39:001
80826994912,9cyclictest131rcu_preempt12:19:181
80826994911,6cyclictest131rcu_preempt10:18:321
8082099494,7cyclictest84376-21cron08:19:590
8082099492,31cyclictest151264-21ssh11:43:030
80820994923,12cyclictest104105-21ssh09:50:190
8082099491,45cyclictest0-21swapper/007:10:010
80820994913,6cyclictest0-21swapper/012:39:200
8082099491,12cyclictest87016-21unin-run09:10:010
80828994811,9cyclictest0-21swapper/208:35:002
8082699488,11cyclictest23-21ksoftirqd/111:08:321
8082699486,6cyclictest131rcu_preempt11:28:471
80826994838,7cyclictest221rcuc/109:34:481
80826994812,8cyclictest131rcu_preempt10:13:181
80826994810,10cyclictest23-21ksoftirqd/111:02:451
80826994810,10cyclictest23-21ksoftirqd/111:02:451
80820994839,6cyclictest16950irq/224-800000010:45:000
80820994839,6cyclictest16950irq/224-800000010:44:590
8082099482,41cyclictest364-21snmpd11:28:210
80831994731,12cyclictest275-21dbus-daemon08:25:023
80831994715,5cyclictest131rcu_preempt10:30:383
8083199471,33cyclictest37-21ksoftirqd/311:50:023
8083199471,33cyclictest37-21ksoftirqd/311:50:023
80826994737,7cyclictest23-21ksoftirqd/109:00:011
80826994734,10cyclictest3467602-21systemd-journal12:10:001
8082699473,39cyclictest221rcuc/111:55:011
80826994728,14cyclictest275-21dbus-daemon12:25:001
80826994714,6cyclictest131rcu_preempt12:05:011
80826994711,9cyclictest131rcu_preempt10:45:511
8082099472,41cyclictest85433-21systemctl08:40:000
80820994722,5cyclictest84913-21latency_hist08:30:020
80820994722,21cyclictest12-21ksoftirqd/011:23:250
80820994721,22cyclictest169259-21ssh12:25:510
80820994719,14cyclictest118719-21ssh10:25:030
8083199466,6cyclictest131rcu_preempt09:25:003
80831994636,7cyclictest37-21ksoftirqd/310:50:483
80831994636,7cyclictest37-21ksoftirqd/310:50:473
80831994615,22cyclictest37-21ksoftirqd/311:01:253
80831994615,22cyclictest37-21ksoftirqd/311:01:243
80831994615,21cyclictest37-21ksoftirqd/310:05:383
80831994615,20cyclictest37-21ksoftirqd/311:44:553
80828994638,5cyclictest82023-21kworker/2:209:00:022
8082699469,9cyclictest131rcu_preempt10:10:011
80826994631,9cyclictest23-21ksoftirqd/110:24:451
80826994628,14cyclictest83843-21sshd08:06:251
80820994616,27cyclictest12-21ksoftirqd/010:56:400
8083199456,26cyclictest131226-21kworker/u8:211:20:023
80831994533,10cyclictest37-21ksoftirqd/308:15:033
80831994526,14cyclictest1-21systemd11:25:013
8083199451,5cyclictest131rcu_preempt11:56:093
80831994515,20cyclictest37-21ksoftirqd/312:22:173
80831994515,20cyclictest37-21ksoftirqd/309:58:103
80831994515,16cyclictest37-21ksoftirqd/309:25:033
80831994514,22cyclictest37-21ksoftirqd/311:27:293
8083199451,25cyclictest0-21swapper/309:39:063
80828994534,6cyclictest83342-21sendmail08:00:002
80828994525,17cyclictest86719-21sshd09:04:232
80826994533,9cyclictest3467602-21systemd-journal07:25:011
80826994533,9cyclictest124124-21sshd10:38:231
80826994528,13cyclictest84898-21sshd08:28:251
8082699452,12cyclictest23-21ksoftirqd/111:30:231
80826994514,8cyclictest131rcu_preempt10:00:011
80820994525,15cyclictest275-21dbus-daemon08:05:010
8082099451,4cyclictest131rcu_preempt09:45:500
8083199448,5cyclictest131rcu_preempt12:32:303
80831994422,18cyclictest37-21ksoftirqd/310:20:373
80831994415,20cyclictest37-21ksoftirqd/309:20:003
80831994414,20cyclictest37-21ksoftirqd/311:35:593
8082899448,8cyclictest0-21swapper/209:10:012
80828994430,10cyclictest30-21ksoftirqd/207:35:022
80828994413,6cyclictest131rcu_preempt08:30:002
8082699448,5cyclictest131rcu_preempt10:40:431
8082699448,5cyclictest131rcu_preempt10:40:421
8082699444,11cyclictest23-21ksoftirqd/111:16:491
80826994429,10cyclictest23-21ksoftirqd/109:25:001
80826994412,5cyclictest131rcu_preempt11:21:181
80826994411,6cyclictest131rcu_preempt07:44:261
80826994411,6cyclictest131rcu_preempt07:44:251
80820994437,4cyclictest16950irq/224-800000008:59:590
80820994437,4cyclictest16850irq/134-800000011:50:000
80820994437,4cyclictest16850irq/134-800000011:50:000
80820994435,7cyclictest12-21ksoftirqd/010:46:280
80820994431,9cyclictest275-21dbus-daemon07:30:000
80820994429,12cyclictest160571-21ssh12:05:040
80820994427,13cyclictest122657-21ssh10:35:010
80820994426,15cyclictest82270-21sshd07:35:020
8083199434,5cyclictest131rcu_preempt08:50:013
80831994327,13cyclictest37-21ksoftirqd/310:17:563
80831994323,14cyclictest3467602-21systemd-journal08:10:013
80831994319,20cyclictest37-21ksoftirqd/309:12:123
80831994316,19cyclictest37-21ksoftirqd/309:48:043
80831994315,17cyclictest37-21ksoftirqd/310:55:163
80831994315,16cyclictest37-21ksoftirqd/310:38:433
80831994314,20cyclictest37-21ksoftirqd/312:28:323
80831994314,20cyclictest37-21ksoftirqd/312:15:473
80831994314,20cyclictest37-21ksoftirqd/311:54:263
80831994314,20cyclictest37-21ksoftirqd/310:28:193
80831994314,20cyclictest37-21ksoftirqd/309:34:043
80831994314,19cyclictest37-21ksoftirqd/312:14:193
80831994314,16cyclictest37-21ksoftirqd/310:41:283
80831994314,16cyclictest37-21ksoftirqd/310:41:283
80831994313,20cyclictest37-21ksoftirqd/312:00:063
80828994325,12cyclictest275-21dbus-daemon07:45:012
80828994325,12cyclictest275-21dbus-daemon07:45:002
80828994312,8cyclictest85462-21/usr/sbin/munin08:40:022
80826994335,4cyclictest152000-21munin-plugin-st11:45:001
80826994332,7cyclictest23-21ksoftirqd/107:30:031
80826994330,10cyclictest23-21ksoftirqd/109:30:021
80826994327,12cyclictest23-21ksoftirqd/107:50:001
80820994338,2cyclictest0-21swapper/008:05:010
80820994329,9cyclictest0-21swapper/009:05:010
80831994232,6cyclictest37-21ksoftirqd/309:05:003
8083199422,6cyclictest131rcu_preempt08:35:313
8083199422,5cyclictest131rcu_preempt08:30:013
80831994215,19cyclictest37-21ksoftirqd/310:13:563
80831994215,18cyclictest37-21ksoftirqd/311:34:283
80831994215,16cyclictest37-21ksoftirqd/312:06:553
80831994214,19cyclictest37-21ksoftirqd/310:47:133
8083199421,15cyclictest0-21swapper/312:38:163
80828994230,8cyclictest30-21ksoftirqd/207:25:012
8082699426,8cyclictest23-21ksoftirqd/109:52:281
80826994230,8cyclictest1-21systemd08:55:001
80826994213,19cyclictest131rcu_preempt11:36:251
8082099421,12cyclictest84677-21/usr/sbin/munin08:25:010
80831994128,10cyclictest3467602-21systemd-journal10:00:353
80831994113,20cyclictest37-21ksoftirqd/309:42:013
80831994113,20cyclictest37-21ksoftirqd/309:42:003
80828994126,11cyclictest86218-21munin-run08:55:022
80826994134,4cyclictest80825-21sshd07:10:021
80826994132,6cyclictest23-21ksoftirqd/112:30:021
80826994132,6cyclictest23-21ksoftirqd/107:40:011
8082699411,6cyclictest0-21swapper/110:26:471
80826994116,18cyclictest23-21ksoftirqd/109:18:441
80820994134,4cyclictest16950irq/224-800000008:15:010
80820994134,4cyclictest16950irq/224-800000008:15:010
8083199406,5cyclictest131rcu_preempt08:00:013
8083199406,5cyclictest131rcu_preempt08:00:003
8083199403,5cyclictest131rcu_preempt08:35:003
80831994015,15cyclictest37-21ksoftirqd/311:05:573
8082699409,6cyclictest131rcu_preempt08:50:011
8083199394,5cyclictest131rcu_preempt07:40:003
80831993930,5cyclictest80996-21cron07:14:593
80831993919,6cyclictest83619-21munin-plugin-st08:05:003
80831993917,18cyclictest37-21ksoftirqd/311:10:563
80828993925,10cyclictest84133-21systemd-run08:15:012
80828993925,10cyclictest84133-21systemd-run08:15:012
80828993923,11cyclictest275-21dbus-daemon08:10:012
80828993920,11cyclictest0-21swapper/208:19:592
8082099392,34cyclictest85686-21sshd08:44:230
80820993923,12cyclictest275-21dbus-daemon08:55:010
80831993826,9cyclictest37-21ksoftirqd/307:55:023
80831993822,11cyclictest3467602-21systemd-journal07:43:253
80831993822,11cyclictest3467602-21systemd-journal07:43:253
8082699386,5cyclictest131rcu_preempt09:00:301
8082699385,9cyclictest23-21ksoftirqd/107:13:241
80826993828,6cyclictest23-21ksoftirqd/108:15:011
80826993828,6cyclictest23-21ksoftirqd/108:15:001
80826993827,7cyclictest83604-21sshd08:04:251
80826993826,9cyclictest23-21ksoftirqd/108:25:001
80826993826,8cyclictest221rcuc/108:35:011
80820993827,7cyclictest83339-21sh08:00:010
80820993827,7cyclictest83339-21sh08:00:010
80820993826,9cyclictest85133-21sshd08:30:230
8082099381,9cyclictest81022-21unin-run07:15:000
8082099381,33cyclictest0-21swapper/007:20:020
80831993723,11cyclictest37-21ksoftirqd/308:20:013
80831993721,12cyclictest3467602-21systemd-journal07:20:013
80828993722,12cyclictest30-21ksoftirqd/207:14:242
8082699379,5cyclictest131rcu_preempt08:18:241
8082699378,8cyclictest23-21ksoftirqd/108:44:241
8082699377,9cyclictest23-21ksoftirqd/107:35:001
80826993721,12cyclictest23-21ksoftirqd/108:39:591
8082099372,3cyclictest131rcu_preempt07:25:010
80831993624,9cyclictest37-21ksoftirqd/307:10:023
8082899367,5cyclictest131rcu_preempt08:05:002
80828993624,9cyclictest84644-21sshd08:23:242
80828993624,9cyclictest0-21swapper/207:10:012
80820993625,8cyclictest12-21ksoftirqd/007:45:010
80820993625,8cyclictest12-21ksoftirqd/007:45:010
8082099361,32cyclictest364-21snmpd07:32:340
80831993522,10cyclictest1-21systemd08:15:003
8082899351,6cyclictest85709-21systemd-run08:45:002
8082099352,29cyclictest12-21ksoftirqd/007:53:250
8082099351,31cyclictest0-21swapper/007:48:550
8083199348,6cyclictest361rcuc/307:50:013
8083199343,4cyclictest131rcu_preempt09:06:123
80831993426,6cyclictest37-21ksoftirqd/308:45:023
8083199342,5cyclictest131rcu_preempt08:58:233
80831993421,10cyclictest37-21ksoftirqd/307:35:013
80831993417,12cyclictest275-21dbus-daemon08:55:013
80826993426,6cyclictest23-21ksoftirqd/107:20:011
8083199336,18cyclictest37-21ksoftirqd/307:29:163
80820993327,4cyclictest0-21swapper/008:47:100
8083199291,17cyclictest37-21ksoftirqd/307:20:413
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional