You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-31 - 22:11

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot5s.osadl.org (updated Sun May 31, 2026 12:44:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
13970993310,22cyclictest13-21ksoftirqd/008:55:000
1397799320,27cyclictest13258-21ssh10:15:132
13977992920,4cyclictest0-21swapper/211:18:292
13981992821,1cyclictest12891-21diskmemload12:18:313
13981992720,1cyclictest1612-21awk10:35:003
1398199271,1cyclictest0-21swapper/311:53:343
1398199270,1cyclictest30948-21/usr/sbin/munin11:33:363
1398199270,0cyclictest0-21swapper/311:10:023
13977992718,5cyclictest8117-21diskmemload12:19:272
13970992725,1cyclictest13-21ksoftirqd/008:30:010
13981992623,2cyclictest0-21swapper/311:04:223
13981992621,4cyclictest0-21swapper/310:44:073
1398199260,2cyclictest605-21avahi-daemon12:15:003
1398199260,21cyclictest0-21swapper/310:45:503
1398199260,1cyclictest0-21swapper/310:40:013
13977992621,4cyclictest8117-21diskmemload09:28:232
13977992621,4cyclictest15732-21ssh11:49:012
13977992620,5cyclictest23345-21perf08:35:012
13977992618,4cyclictest0-21swapper/211:27:282
13972992624,1cyclictest0-21swapper/112:03:351
1397299260,25cyclictest5035-21rm11:38:461
1397099265,21cyclictest13-21ksoftirqd/012:09:030
1397099261,24cyclictest26761-21sh11:29:180
1397099260,25cyclictest560-21ssh11:04:390
13981992523,2cyclictest0-21swapper/311:42:283
13981992520,1cyclictest0-21swapper/309:27:393
13981992520,1cyclictest0-21swapper/309:22:283
1398199250,21cyclictest4391-21ssh12:08:593
1398199250,21cyclictest30770-21ssh09:31:473
1398199250,1cyclictest25270-21ssh10:27:253
1398199250,1cyclictest17641-21diskmemload12:22:383
1398199250,1cyclictest17641-21diskmemload12:22:383
1398199250,1cyclictest0-21swapper/311:56:243
13977992523,1cyclictest0-21swapper/212:28:332
13977992522,3cyclictest0-21swapper/211:14:492
1397799250,24cyclictest0-21swapper/212:34:562
1397799250,24cyclictest0-21swapper/212:10:172
1397799250,24cyclictest0-21swapper/210:36:502
1397799250,24cyclictest0-21swapper/209:44:192
1397799250,21cyclictest11862-21grep09:13:372
1397099250,24cyclictest0-21swapper/011:11:270
13981992422,2cyclictest0-21swapper/309:19:093
13981992421,2cyclictest8117-21diskmemload11:45:303
1398199240,23cyclictest23871-21apt-get08:38:323
1398199240,23cyclictest18565-21grep08:25:013
1398199240,23cyclictest0-21swapper/309:48:313
1398199240,21cyclictest8117-21diskmemload10:58:013
1398199240,20cyclictest0-21swapper/310:07:163
1398199240,0cyclictest0-21swapper/311:22:133
13977992422,1cyclictest0-21swapper/209:32:292
13977992420,3cyclictest0-21swapper/212:05:372
13977992420,3cyclictest0-21swapper/210:47:492
13977992420,3cyclictest0-21swapper/210:32:172
1397799240,23cyclictest8117-21diskmemload11:54:402
1397799240,23cyclictest8117-21diskmemload11:22:582
1397799240,23cyclictest0-21swapper/211:38:022
1397799240,23cyclictest0-21swapper/209:45:302
1397799240,23cyclictest0-21swapper/209:19:252
1397799240,21cyclictest0-21swapper/210:27:062
1397799240,0cyclictest8117-21diskmemload11:33:332
1397799240,0cyclictest0-21swapper/211:08:472
1397799240,0cyclictest0-21swapper/210:51:012
13972992424,0cyclictest0-21swapper/110:19:141
13972992423,1cyclictest8117-21diskmemload09:35:251
13972992423,1cyclictest6969-21ssh10:39:591
13972992421,2cyclictest7785-21sh10:40:591
1397299240,23cyclictest9603-21ssh09:11:511
1397299240,0cyclictest0-21swapper/111:00:471
1397299240,0cyclictest0-21swapper/110:20:181
13970992423,1cyclictest8117-21diskmemload10:25:220
13970992422,1cyclictest4063-21cron07:55:000
13970992422,1cyclictest19317-21sh09:51:000
1397099240,24cyclictest0-21swapper/010:47:020
1397099240,23cyclictest29138-21munin-run10:00:010
1397099240,23cyclictest13-21ksoftirqd/007:45:020
1397099240,23cyclictest0-21swapper/011:21:580
1397099240,23cyclictest0-21swapper/010:03:410
1397099240,1cyclictest8117-21diskmemload12:12:360
13981992323,0cyclictest8117-21diskmemload10:51:173
13981992323,0cyclictest0-21swapper/310:19:403
1398199230,23cyclictest0-21swapper/309:59:343
1398199230,22cyclictest8117-21diskmemload09:10:203
1398199230,22cyclictest4411-21sh09:38:013
1398199230,22cyclictest30596-21rm10:01:563
1398199230,22cyclictest29924-21ssh12:33:393
1398199230,22cyclictest21510-21ssh12:25:363
1398199230,22cyclictest16082-21turbostat.cron08:20:013
1398199230,22cyclictest1143-21ssh11:05:153
1398199230,22cyclictest0-21swapper/309:40:433
1398199230,22cyclictest0-21swapper/307:53:313
1398199230,20cyclictest0-21swapper/312:01:033
1398199230,0cyclictest8117-21diskmemload09:51:143
13977992323,0cyclictest8117-21diskmemload10:06:532
13977992322,1cyclictest0-21swapper/210:56:202
13977992321,2cyclictest0-21swapper/211:04:112
13977992321,1cyclictest9415-21apt-get08:08:302
13977992320,3cyclictest0-21swapper/210:02:252
13977992320,3cyclictest0-21swapper/209:36:282
13977992318,4cyclictest24125-21apt-get07:33:322
1397799231,21cyclictest32034-1kworker/u9:010:10:062
1397799230,23cyclictest0-21swapper/210:20:332
1397799230,23cyclictest0-21swapper/209:55:282
1397799230,23cyclictest0-21swapper/209:24:062
1397799230,22cyclictest2211-21apt-get07:53:322
1397799230,0cyclictest0-21swapper/212:01:102
1397799230,0cyclictest0-21swapper/210:40:322
1397799230,0cyclictest0-21swapper/209:51:382
13972992323,0cyclictest8117-21diskmemload09:29:021
13972992323,0cyclictest0-21swapper/109:56:011
13972992323,0cyclictest0-21swapper/109:18:221
13972992321,1cyclictest23910-21sh11:27:321
13972992321,1cyclictest0-21swapper/110:55:011
13972992320,2cyclictest17246-21rm10:49:431
1397299230,2cyclictest24272-21sh10:25:501
1397299230,2cyclictest23475-21ssh10:55:501
1397299230,2cyclictest1057-21rm10:34:071
1397299230,23cyclictest0-21swapper/109:45:291
1397299230,22cyclictest878-21timerwakeupswit10:03:411
1397299230,22cyclictest6955-21sh11:11:031
1397299230,22cyclictest23883-21rm11:57:561
1397299230,22cyclictest21945-21ssh12:26:141
1397299230,22cyclictest17890-21ssh11:51:491
1397299230,22cyclictest0-21swapper/112:06:121
1397299230,0cyclictest8117-21diskmemload11:46:581
1397299230,0cyclictest0-21swapper/112:30:161
1397299230,0cyclictest0-21swapper/109:20:281
13970992323,0cyclictest8117-21diskmemload09:12:560
13970992323,0cyclictest0-21swapper/012:38:020
13970992322,1cyclictest62550irq/127-eno112:21:470
13970992322,1cyclictest62550irq/127-eno112:21:470
13970992322,1cyclictest62550irq/127-eno111:15:210
13970992321,1cyclictest23839-21ssh11:57:530
13970992320,2cyclictest8237-21ssh10:11:100
13970992320,2cyclictest0-21swapper/009:32:260
1397099231,21cyclictest62550irq/127-eno112:25:170
1397099230,2cyclictest3855-21basename11:38:340
1397099230,23cyclictest0-21swapper/011:07:550
1397099230,22cyclictest8861-21ssh10:42:400
1397099230,22cyclictest8117-21diskmemload10:07:110
1397099230,22cyclictest30294-21ssh10:32:010
1397099230,22cyclictest2952-21ssh09:35:480
1397099230,22cyclictest27233-21ssh10:58:500
1397099230,22cyclictest15236-21ssh09:17:140
1397099230,22cyclictest12888-21ssh11:47:220
1397099230,22cyclictest10620-21ssh12:15:010
1397099230,22cyclictest0-21swapper/009:47:540
1397099230,1cyclictest0-21swapper/011:40:160
1397099230,0cyclictest8117-21diskmemload11:54:570
1397099230,0cyclictest0-21swapper/010:35:350
1397099230,0cyclictest0-21swapper/009:24:010
13981992220,1cyclictest1058-21ssh11:35:303
1398199221,1cyclictest0-21swapper/308:10:003
1398199220,22cyclictest3462-21diskmemload12:38:483
1398199220,21cyclictest7024-21apt-get08:03:323
1398199220,21cyclictest19280-21ssh10:21:183
1398199220,21cyclictest10770-21memory10:13:373
1398199220,21cyclictest0-21swapper/311:28:333
1398199220,1cyclictest13688-21awk08:15:003
13977992222,0cyclictest0-21swapper/212:22:222
13977992222,0cyclictest0-21swapper/212:22:222
13977992220,2cyclictest0-21swapper/211:55:032
1397799220,22cyclictest8117-21diskmemload12:35:022
1397799220,21cyclictest622-21gdbus08:50:002
1397799220,21cyclictest0-21swapper/211:40:312
13972992222,0cyclictest0-21swapper/112:10:201
13972992222,0cyclictest0-21swapper/109:52:001
13972992222,0cyclictest0-21swapper/107:58:321
13972992220,2cyclictest871-21ssh12:37:261
13972992220,1cyclictest8235-21rm09:40:451
13972992220,1cyclictest0-21swapper/110:05:031
13972992220,1cyclictest0-21swapper/107:45:021
13972992219,2cyclictest28611-21ssh11:31:061
1397299220,21cyclictest8117-21diskmemload12:15:051
1397299220,21cyclictest0-21swapper/108:53:321
1397299220,1cyclictest8117-21diskmemload11:06:051
1397299220,1cyclictest17221-21ssh11:20:231
1397299220,1cyclictest12325-21ssh11:16:051
1397299220,1cyclictest0-21swapper/112:21:561
1397299220,1cyclictest0-21swapper/112:21:561
13970992222,0cyclictest13-21ksoftirqd/011:31:530
13970992222,0cyclictest0-21swapper/012:30:460
13970992221,1cyclictest141rcuc/012:02:510
13970992220,1cyclictest28532-21dpkg07:40:010
13970992220,1cyclictest18554-21rm10:20:090
13970992220,1cyclictest0-21swapper/009:25:320
13970992219,2cyclictest31279-21cron07:45:010
1397099220,21cyclictest6271-21apt-get09:08:320
1397099220,21cyclictest0-21swapper/008:18:320
1397099220,1cyclictest8203-21ssh09:40:420
1397099220,1cyclictest18807-21perf07:20:000
1397099220,1cyclictest13598-21ssh10:15:460
13981992120,0cyclictest0-21swapper/307:48:313
1398199210,20cyclictest0-21swapper/308:58:313
1398199210,20cyclictest0-21swapper/308:48:323
1398199210,20cyclictest0-21swapper/307:18:303
1398199210,0cyclictest4605-21apt-get07:58:323
1398199210,0cyclictest0-21swapper/308:53:323
13977992120,0cyclictest11811-21apt-get08:13:322
13977992119,1cyclictest16631-21apt-get08:23:322
1397799210,20cyclictest0-21swapper/207:38:312
1397799210,1cyclictest0-21swapper/209:00:002
1397799210,0cyclictest0-21swapper/208:28:342
13972992120,1cyclictest28501-21apt-get07:40:021
13970992121,0cyclictest0-21swapper/008:03:320
1397099210,21cyclictest0-21swapper/010:50:500
1397099210,21cyclictest0-21swapper/008:08:300
13981992019,1cyclictest19334-21apt-get07:23:313
13977992020,0cyclictest14466-21apt-get07:13:312
13977992020,0cyclictest0-21swapper/207:43:332
13977992018,1cyclictest8122-21ssh09:10:012
1397799200,19cyclictest0-21swapper/209:03:322
1397799200,0cyclictest0-21swapper/208:03:322
13972992020,0cyclictest26523-21apt-get07:38:321
13972992019,0cyclictest0-21swapper/108:48:321
1397299200,0cyclictest0-21swapper/108:28:341
1397299200,0cyclictest0-21swapper/108:18:311
1397299200,0cyclictest0-21swapper/107:13:311
13970992020,0cyclictest0-21swapper/007:23:310
13970992018,1cyclictest23574-21cron07:30:010
1397099200,0cyclictest28152-21perf08:45:000
1397099200,0cyclictest21478-21apt-get08:33:310
1397099200,0cyclictest0-21swapper/008:38:310
13981991917,1cyclictest0-21swapper/307:40:023
13977991919,0cyclictest0-21swapper/207:23:312
13977991916,2cyclictest16076-21apt08:20:012
13970991919,0cyclictest0-21swapper/007:13:310
13970991918,0cyclictest0-21swapper/008:48:320
13981991818,0cyclictest0-21swapper/308:43:323
13977991817,1cyclictest21728-21apt-get07:28:312
1397299180,18cyclictest0-21swapper/107:18:301
13970991818,0cyclictest0-21swapper/008:55:020
13981991717,0cyclictest0-21swapper/309:03:323
13977991717,0cyclictest0-21swapper/207:18:302
13977991715,1cyclictest26287-21apt-get08:43:312
13972991717,0cyclictest0-21swapper/107:33:321
13972991716,1cyclictest23871-21apt-get08:38:311
13981991616,0cyclictest0-21swapper/307:38:323
13981991616,0cyclictest0-21swapper/307:13:313
13977991616,0cyclictest32225-21apt-get07:48:312
13977991615,0cyclictest0-21swapper/208:38:322
13972991616,0cyclictest0-21swapper/108:08:301
1397299160,15cyclictest4400-21perl09:03:341
13970991616,0cyclictest0-21swapper/007:58:320
13970991614,1cyclictest141rcuc/008:23:400
1397099160,15cyclictest13295-21cat08:13:400
13981991515,0cyclictest0-21swapper/308:33:323
13977991515,0cyclictest0-21swapper/207:58:322
13981991414,0cyclictest0-21swapper/309:05:233
13981991414,0cyclictest0-21swapper/307:30:023
1398199140,14cyclictest0-21swapper/308:25:263
1398199140,14cyclictest0-21swapper/307:25:013
1397299140,14cyclictest0-21swapper/108:55:111
1397299140,14cyclictest0-21swapper/108:30:201
1397299140,14cyclictest0-21swapper/108:20:571
1397299140,14cyclictest0-21swapper/108:10:291
1397299140,14cyclictest0-21swapper/107:50:111
1397299140,14cyclictest0-21swapper/107:26:291
1397299140,14cyclictest0-21swapper/107:20:591
1397299140,0cyclictest0-21swapper/108:00:241
13970991414,0cyclictest0-21swapper/009:00:330
1397099140,14cyclictest0-21swapper/007:30:300
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional