You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-10 - 18:27

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot3s.osadl.org (updated Thu Sep 10, 2026 12:45:24)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
215989991229,4cyclictest131rcu_preempt09:35:012
215980997549,23cyclictest12-21ksoftirqd/012:04:020
215989997152,15cyclictest3467602-21systemd-journal10:00:012
21598099687,4cyclictest12-21ksoftirqd/009:10:020
21598099687,4cyclictest12-21ksoftirqd/009:10:010
215980996722,38cyclictest12-21ksoftirqd/010:05:030
21598099632,3cyclictest131rcu_preempt11:41:550
215989996151,7cyclictest209883-21kworker/2:210:20:032
215980996132,10cyclictest3467602-21systemd-journal12:09:240
215980996114,4cyclictest12-21ksoftirqd/011:35:020
215985996020,5cyclictest131rcu_preempt11:13:311
21598099603,34cyclictest131rcu_preempt12:19:080
215980996031,14cyclictest291187-21ssh11:38:040
21598099602,39cyclictest131rcu_preempt12:10:270
215980996023,33cyclictest1-21systemd12:25:020
215980996023,33cyclictest1-21systemd12:25:020
215985995915,5cyclictest131rcu_preempt11:40:021
215980995923,32cyclictest269779-21sh10:51:540
215980995919,34cyclictest16850irq/134-800000011:55:020
215980995917,33cyclictest16850irq/134-800000012:35:580
21598099581,41cyclictest314096-21ssh12:27:070
215980995812,20cyclictest131rcu_preempt08:50:010
215989995736,13cyclictest1-21systemd11:55:012
21598999571,5cyclictest3467718-21lldpd09:21:212
215985995714,6cyclictest131rcu_preempt09:50:021
215985995710,7cyclictest131rcu_preempt07:20:021
215993995610,6cyclictest131rcu_preempt08:25:003
215989995648,5cyclictest240856-21kworker/2:010:44:112
21598599567,10cyclictest23-21ksoftirqd/112:40:021
215980995611,41cyclictest12-21ksoftirqd/009:17:340
215989995546,5cyclictest240856-21kworker/2:010:27:192
215989995546,5cyclictest240856-21kworker/2:010:27:192
215989995536,14cyclictest3467602-21systemd-journal08:30:002
215985995536,15cyclictest3467602-21systemd-journal07:30:011
215985995536,15cyclictest3467602-21systemd-journal07:30:001
215985995523,12cyclictest23-21ksoftirqd/109:51:191
215980995529,11cyclictest286699-21ssh11:28:200
215980995528,12cyclictest295522-21ssh11:47:180
215980995526,14cyclictest283308-21ssh11:20:470
215989995445,6cyclictest274296-21kworker/2:111:47:022
215989995438,12cyclictest3467602-21systemd-journal09:00:012
215989995438,12cyclictest3467602-21systemd-journal09:00:002
215985995418,6cyclictest131rcu_preempt10:15:021
215980995429,11cyclictest252488-21ssh10:14:580
215980995413,36cyclictest0-21swapper/011:01:460
215993995337,12cyclictest3467602-21systemd-journal11:40:003
215989995345,5cyclictest209883-21kworker/2:209:37:122
215989995344,5cyclictest240856-21kworker/2:010:46:262
215985995312,6cyclictest131rcu_preempt10:10:021
21598099534,5cyclictest131rcu_preempt10:16:120
215980995323,15cyclictest243211-21ssh09:55:010
215980995318,31cyclictest16850irq/134-800000009:26:370
215980995316,20cyclictest16850irq/134-800000011:10:110
215980995315,4cyclictest12-21ksoftirqd/009:48:320
21598099531,37cyclictest244019-21ssh09:56:270
215993995215,6cyclictest131rcu_preempt10:00:013
215989995244,5cyclictest240856-21kworker/2:011:04:062
215989995242,6cyclictest274296-21kworker/2:111:27:102
215985995211,6cyclictest131rcu_preempt12:04:291
215985995210,6cyclictest131rcu_preempt12:33:231
215985995210,5cyclictest131rcu_preempt11:27:331
215985995210,11cyclictest23-21ksoftirqd/110:46:281
21598099522,9cyclictest121ksoftirqd/010:42:250
215980995220,16cyclictest317145-21ssh12:33:450
215980995217,31cyclictest16850irq/134-800000009:35:430
215980995216,32cyclictest16850irq/134-800000011:07:030
215980995216,32cyclictest16850irq/134-800000010:48:220
215980995216,32cyclictest16850irq/134-800000010:30:030
215980995216,32cyclictest16850irq/134-800000010:30:030
215980995216,32cyclictest16850irq/134-800000010:02:400
215980995216,32cyclictest16850irq/134-800000010:02:390
215980995215,33cyclictest16850irq/134-800000010:35:020
215993995129,16cyclictest275-21dbus-daemon09:50:023
215989995143,5cyclictest274296-21kworker/2:112:15:022
215989995143,5cyclictest240856-21kworker/2:010:10:442
215989995142,6cyclictest240856-21kworker/2:011:15:012
215989995142,5cyclictest222172-21kworker/2:109:27:412
215989995132,14cyclictest275-21dbus-daemon11:35:012
21598599517,18cyclictest131rcu_preempt11:50:011
215985995136,12cyclictest1-21systemd12:00:011
215985995115,7cyclictest23-21ksoftirqd/109:45:021
215985995113,6cyclictest131rcu_preempt09:40:021
215985995110,6cyclictest131rcu_preempt12:20:481
215985995110,6cyclictest131rcu_preempt12:20:471
21598099519,18cyclictest221349-21kworker/u8:109:00:030
21598099519,18cyclictest221349-21kworker/u8:109:00:020
21598099518,29cyclictest16850irq/134-800000009:10:120
215980995124,12cyclictest301215-21ssh11:59:480
215980995115,32cyclictest16850irq/134-800000010:21:010
21599399506,5cyclictest131rcu_preempt09:45:033
215989995043,4cyclictest222172-21kworker/2:109:14:292
215989995042,5cyclictest274296-21kworker/2:111:20:292
215989995042,5cyclictest240856-21kworker/2:010:52:132
215989995041,6cyclictest222172-21kworker/2:109:41:322
215989995034,12cyclictest3467602-21systemd-journal07:30:012
215989995034,12cyclictest3467602-21systemd-journal07:30:002
21598999502,3cyclictest131rcu_preempt11:08:262
215980995042,4cyclictest16950irq/224-800000010:40:010
215980995037,8cyclictest12-21ksoftirqd/008:39:590
215980995023,12cyclictest231888-21ssh09:30:320
215980995015,20cyclictest16850irq/134-800000009:22:000
215980995014,21cyclictest16850irq/134-800000009:41:500
21599399499,5cyclictest131rcu_preempt12:16:463
21599399493,5cyclictest131rcu_preempt09:40:013
21599399491,9cyclictest218783-21munin-plugin-st08:04:593
215993994914,22cyclictest37-21ksoftirqd/309:50:333
215989994941,5cyclictest274296-21kworker/2:111:19:082
215989994941,5cyclictest209883-21kworker/2:210:05:302
215989994941,5cyclictest209883-21kworker/2:209:54:442
215989994940,6cyclictest240856-21kworker/2:011:55:372
215985994920,9cyclictest131rcu_preempt12:06:401
215985994914,5cyclictest131rcu_preempt11:50:341
215985994913,6cyclictest131rcu_preempt11:20:011
215980994915,15cyclictest131rcu_preempt08:15:010
21599399487,6cyclictest131rcu_preempt10:02:373
21599399487,6cyclictest131rcu_preempt10:02:363
21599399482,4cyclictest131rcu_preempt10:48:243
215993994815,24cyclictest37-21ksoftirqd/311:00:433
21599399481,43cyclictest299807-21sshd11:56:243
215993994811,5cyclictest131rcu_preempt09:20:013
215989994841,4cyclictest274296-21kworker/2:112:34:232
215989994840,5cyclictest274296-21kworker/2:112:27:292
215989994840,5cyclictest274296-21kworker/2:111:44:352
215989994840,5cyclictest209883-21kworker/2:209:03:242
215989994840,5cyclictest209883-21kworker/2:208:05:022
21598999481,4cyclictest131rcu_preempt10:37:012
21598599488,7cyclictest23-21ksoftirqd/110:35:491
21598599486,5cyclictest131rcu_preempt11:21:571
215985994826,7cyclictest131rcu_preempt09:22:281
215985994812,8cyclictest131rcu_preempt09:58:091
215985994811,6cyclictest131rcu_preempt11:01:571
215985994811,6cyclictest131rcu_preempt10:42:001
21598099481,44cyclictest0-21swapper/008:55:020
215980994811,4cyclictest12-21ksoftirqd/011:20:020
215993994715,22cyclictest37-21ksoftirqd/311:16:203
215993994715,22cyclictest37-21ksoftirqd/310:27:333
215993994715,22cyclictest37-21ksoftirqd/310:27:323
215993994715,20cyclictest37-21ksoftirqd/310:19:213
215993994715,19cyclictest37-21ksoftirqd/310:36:033
215993994715,19cyclictest37-21ksoftirqd/309:27:223
215989994739,5cyclictest209883-21kworker/2:210:20:032
21598599479,8cyclictest131rcu_preempt12:20:021
21598599473,6cyclictest131rcu_preempt10:25:031
215985994717,19cyclictest23-21ksoftirqd/110:26:321
215985994717,19cyclictest23-21ksoftirqd/110:26:311
215985994713,5cyclictest131rcu_preempt08:25:011
215980994719,26cyclictest12-21ksoftirqd/007:57:460
215980994712,21cyclictest16850irq/134-800000010:57:010
215993994615,19cyclictest37-21ksoftirqd/310:52:533
215993994615,17cyclictest37-21ksoftirqd/312:05:353
215993994615,15cyclictest37-21ksoftirqd/311:48:063
215993994614,23cyclictest37-21ksoftirqd/312:38:593
215993994614,23cyclictest37-21ksoftirqd/309:12:553
215993994614,22cyclictest37-21ksoftirqd/311:42:483
215993994614,22cyclictest37-21ksoftirqd/311:05:143
215993994614,17cyclictest37-21ksoftirqd/311:50:333
215993994613,22cyclictest37-21ksoftirqd/312:25:073
215993994613,21cyclictest37-21ksoftirqd/312:30:343
215989994638,5cyclictest274296-21kworker/2:112:35:032
215989994637,5cyclictest209883-21kworker/2:210:02:072
215989994637,5cyclictest209883-21kworker/2:210:02:062
215989994636,6cyclictest215585-21kworker/2:308:37:002
215989994630,12cyclictest273534-21cut11:00:012
21598599469,6cyclictest131rcu_preempt12:30:021
21598599469,6cyclictest131rcu_preempt12:14:531
215985994628,14cyclictest275-21dbus-daemon08:30:001
21598099464,37cyclictest220551-21kworker/u8:008:45:020
21598099463,6cyclictest131rcu_preempt07:50:010
21598099461,40cyclictest220058-21sshd08:28:250
21599399453,5cyclictest131rcu_preempt07:30:033
21599399453,5cyclictest131rcu_preempt07:30:023
215993994523,13cyclictest301658-21ssh12:00:123
215993994515,20cyclictest37-21ksoftirqd/310:21:423
215993994515,15cyclictest37-21ksoftirqd/309:32:443
215993994515,10cyclictest37-21ksoftirqd/311:12:233
215993994514,21cyclictest37-21ksoftirqd/310:12:213
215993994514,21cyclictest37-21ksoftirqd/310:09:203
215989994539,3cyclictest274296-21kworker/2:112:21:102
215989994539,3cyclictest274296-21kworker/2:112:21:092
215989994538,5cyclictest274296-21kworker/2:111:38:422
215989994537,5cyclictest209883-21kworker/2:209:46:092
215989994511,6cyclictest131rcu_preempt09:20:002
215985994535,6cyclictest23-21ksoftirqd/109:19:571
215980994536,6cyclictest16950irq/224-800000007:20:010
215993994429,12cyclictest218186-21sed07:50:023
215993994416,17cyclictest37-21ksoftirqd/310:40:543
215993994415,14cyclictest37-21ksoftirqd/312:24:573
215993994415,14cyclictest37-21ksoftirqd/312:24:563
215993994414,18cyclictest37-21ksoftirqd/312:10:053
215993994410,5cyclictest131rcu_preempt07:40:003
215989994436,5cyclictest209883-21kworker/2:207:50:032
215989994434,6cyclictest209883-21kworker/2:207:16:442
21598599447,6cyclictest131rcu_preempt11:30:081
21598599446,5cyclictest131rcu_preempt10:32:591
215985994414,6cyclictest131rcu_preempt10:05:011
215985994414,6cyclictest131rcu_preempt10:05:001
215985994411,5cyclictest131rcu_preempt11:08:321
215985994410,6cyclictest131rcu_preempt10:55:481
215985994410,5cyclictest131rcu_preempt10:50:021
215985994410,5cyclictest131rcu_preempt09:29:411
215980994437,4cyclictest16950irq/224-800000008:04:590
215980994424,14cyclictest275-21dbus-daemon09:05:010
215980994417,20cyclictest131rcu_preempt07:55:000
215993994315,17cyclictest37-21ksoftirqd/310:30:463
215993994314,14cyclictest37-21ksoftirqd/311:25:303
215993994312,19cyclictest37-21ksoftirqd/311:20:443
215993994311,20cyclictest37-21ksoftirqd/309:23:333
215989994335,5cyclictest274296-21kworker/2:112:16:172
215989994335,5cyclictest240856-21kworker/2:012:02:522
215989994328,12cyclictest275-21dbus-daemon07:45:012
215989994326,12cyclictest275-21dbus-daemon07:15:012
21598999431,31cyclictest0-21swapper/209:10:022
21598999431,31cyclictest0-21swapper/209:10:012
21598999431,10cyclictest217235-21unin-run07:35:002
21598599439,5cyclictest131rcu_preempt09:31:161
21598599434,6cyclictest131rcu_preempt09:14:591
215985994322,5cyclictest23-21ksoftirqd/111:44:331
215985994313,5cyclictest131rcu_preempt08:00:011
215985994312,11cyclictest23-21ksoftirqd/110:15:221
215980994332,8cyclictest12-21ksoftirqd/007:30:020
215980994332,8cyclictest12-21ksoftirqd/007:30:020
215980994321,18cyclictest12-21ksoftirqd/008:10:010
21598099431,5cyclictest131rcu_preempt07:15:000
21599399424,6cyclictest131rcu_preempt07:15:003
21599399422,5cyclictest131rcu_preempt09:00:013
21599399422,5cyclictest131rcu_preempt09:00:013
215993994220,7cyclictest37-21ksoftirqd/310:56:103
215993994213,20cyclictest37-21ksoftirqd/311:33:453
215989994234,5cyclictest209883-21kworker/2:210:32:472
215989994233,5cyclictest240856-21kworker/2:012:05:452
215989994227,9cyclictest275-21dbus-daemon08:45:022
215989994225,14cyclictest30-21ksoftirqd/208:15:022
215989994225,13cyclictest217483-21systemctl07:40:012
215989994223,12cyclictest275-21dbus-daemon08:55:012
21598599429,5cyclictest131rcu_preempt08:45:001
215985994232,7cyclictest23-21ksoftirqd/107:38:111
215985994211,4cyclictest131rcu_preempt07:46:241
215985994211,24cyclictest217741-21modprobe07:45:011
21599399416,5cyclictest131rcu_preempt08:20:023
21599399416,5cyclictest131rcu_preempt08:20:013
215993994119,5cyclictest37-21ksoftirqd/307:57:473
21598999419,6cyclictest131rcu_preempt07:58:232
215989994133,5cyclictest215585-21kworker/2:308:30:572
215989994133,5cyclictest209883-21kworker/2:207:24:472
21598599418,6cyclictest131rcu_preempt08:35:031
21598599418,5cyclictest131rcu_preempt09:05:341
21598599418,5cyclictest131rcu_preempt09:05:341
215985994111,7cyclictest131rcu_preempt08:50:241
215985994111,5cyclictest131rcu_preempt08:15:021
215980994134,4cyclictest16950irq/224-800000007:40:010
21599399408,6cyclictest131rcu_preempt08:50:023
215993994017,10cyclictest37-21ksoftirqd/309:06:533
215993994017,10cyclictest37-21ksoftirqd/309:06:533
215993994013,16cyclictest37-21ksoftirqd/308:10:093
21598599403,6cyclictest131rcu_preempt09:00:011
21598599403,6cyclictest131rcu_preempt09:00:011
215985994023,12cyclictest23-21ksoftirqd/109:05:011
215985994021,4cyclictest23-21ksoftirqd/108:37:561
215985994015,6cyclictest131rcu_preempt07:35:011
21598099407,25cyclictest215984-21wc07:10:000
215980994019,15cyclictest275-21dbus-daemon07:25:000
21598099401,7cyclictest219538-21modprobe08:20:000
21598099401,7cyclictest219538-21modprobe08:20:000
215985993925,11cyclictest23-21ksoftirqd/107:25:001
21598099396,5cyclictest131rcu_preempt08:35:020
215993993816,4cyclictest37-21ksoftirqd/308:30:013
21598999389,6cyclictest131rcu_preempt08:20:032
21598999389,6cyclictest131rcu_preempt08:20:022
215989993829,5cyclictest209883-21kworker/2:207:50:082
21598599389,5cyclictest131rcu_preempt07:15:021
215985993827,8cyclictest218936-21sed08:05:011
215985993810,5cyclictest131rcu_preempt08:19:301
215985993810,5cyclictest131rcu_preempt08:19:301
215993993727,7cyclictest217733-21sshd07:44:243
215993993721,12cyclictest275-21dbus-daemon08:40:003
21598599378,6cyclictest131rcu_preempt07:55:001
215985993729,5cyclictest219041-21sh08:10:001
215980993717,17cyclictest12-21ksoftirqd/007:34:580
215980993715,4cyclictest12-21ksoftirqd/008:25:010
215993993623,9cyclictest309-21in:imuxsock07:18:253
21598999367,5cyclictest131rcu_preempt08:45:242
215989993625,8cyclictest3467602-21systemd-journal08:09:592
21598599360,4cyclictest131rcu_preempt07:10:021
215980993623,9cyclictest12-21ksoftirqd/007:43:570
21599399353,5cyclictest131rcu_preempt08:45:013
215993993526,5cyclictest220329-21munin-plugin-st08:35:013
215993993523,9cyclictest37-21ksoftirqd/308:54:233
215993993517,4cyclictest37-21ksoftirqd/308:07:333
215993993517,4cyclictest37-21ksoftirqd/307:50:273
215993993421,9cyclictest275-21dbus-daemon07:35:013
215993993418,12cyclictest275-21dbus-daemon09:05:003
21599399341,5cyclictest131rcu_preempt07:25:013
215989993422,10cyclictest30-21ksoftirqd/208:23:242
21598599345,8cyclictest23-21ksoftirqd/108:45:241
215993993326,4cyclictest216091-21latency_hist07:10:023
21598999332,3cyclictest131rcu_preempt07:10:002
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional