You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-13 - 07:06

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot7s.osadl.org (updated Sat Jun 13, 2026 00:43:48)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2950029480,9sleep00-21swapper/019:06:210
2953429076,10sleep20-21swapper/219:06:472
2965528667,15sleep10-21swapper/119:08:231
2956028269,9sleep30-21swapper/319:07:093
306812640,1sleep030682-21gzip00:08:220
2992399331,31cyclictest0-21swapper/022:10:110
29926992928,1cyclictest0-21swapper/121:52:461
29926992928,1cyclictest0-21swapper/121:52:451
2992399292,25cyclictest31929-21turbostat.cron20:20:000
2992399291,27cyclictest1228-21fgrep23:50:210
29926992825,2cyclictest38150irq/131-snd_hda22:15:101
2992399274,22cyclictest23292-21diskmemload23:38:070
2992399272,24cyclictest23292-21diskmemload00:16:080
2992399270,26cyclictest0-21swapper/023:45:120
2992399264,21cyclictest23292-21diskmemload22:50:090
2992399263,18cyclictest23292-21diskmemload21:12:340
2992399260,1cyclictest23292-21diskmemload23:55:130
2992399260,0cyclictest0-21swapper/021:20:260
2993099250,24cyclictest0-21swapper/221:44:082
2992399253,21cyclictest23292-21diskmemload00:39:440
2992399250,1cyclictest0-21swapper/023:05:270
2992399250,13cyclictest0-21swapper/022:58:150
2992399250,0cyclictest0-21swapper/022:29:040
2992399250,0cyclictest0-21swapper/021:17:230
77742240,0sleep00-21swapper/000:13:590
2992399242,2cyclictest23292-21diskmemload23:31:370
2992399242,21cyclictest23292-21diskmemload22:23:370
2992399241,20cyclictest21417-21tr21:05:150
2993099230,23cyclictest0-21swapper/200:06:272
2993099230,22cyclictest0-21swapper/220:55:262
2992699230,22cyclictest0-21swapper/123:50:121
2992399233,19cyclictest0-21swapper/022:46:420
2992399232,1cyclictest23292-21diskmemload23:41:410
2992399232,19cyclictest23292-21diskmemload23:14:170
2992399232,16cyclictest23292-21diskmemload23:17:550
2992399231,1cyclictest27896-21apt-get22:30:100
2992399230,1cyclictest0-21swapper/000:30:250
203402230,0chrt0-21swapper/123:42:431
2993099220,21cyclictest0-21swapper/221:15:312
2992399223,18cyclictest23292-21diskmemload22:44:150
2992399222,1cyclictest23292-21diskmemload22:18:370
2992399221,2cyclictest12184-21munin-run19:40:000
2992399221,20cyclictest24299-21turbostat00:05:000
2993099210,20cyclictest0-21swapper/222:31:092
2992699210,1cyclictest0-21swapper/123:06:211
2992399212,18cyclictest0-21swapper/021:45:140
2992399210,20cyclictest23292-21diskmemload21:30:450
2992399210,1cyclictest0-21swapper/021:36:340
2993399200,20cyclictest0-21swapper/321:13:043
2993099201,19cyclictest0-21swapper/200:34:142
2993099200,20cyclictest0-21swapper/221:20:232
2993099200,0cyclictest0-21swapper/221:34:182
2992399202,1cyclictest0-21swapper/021:27:300
2992399201,1cyclictest23292-21diskmemload22:07:390
2992399201,1cyclictest23292-21diskmemload00:25:320
2992399200,2cyclictest0-21swapper/000:20:530
2992399200,19cyclictest0-21swapper/023:22:530
2993399190,1cyclictest0-21swapper/321:49:333
29930991916,1cyclictest0-21swapper/223:48:572
2993099190,1cyclictest0-21swapper/222:28:152
29926991918,1cyclictest0-21swapper/122:25:311
29926991917,1cyclictest29883-21apache223:28:591
2992399192,1cyclictest23292-21diskmemload21:41:560
2992399190,3cyclictest0-21swapper/022:01:400
2992399190,2cyclictest0-21swapper/021:58:250
2992399190,2cyclictest0-21swapper/021:58:250
2992399190,1cyclictest0-21swapper/023:04:030
2993099181,2cyclictest141rcu_preempt23:27:332
2993099180,3cyclictest0-21swapper/221:56:412
2993099180,3cyclictest0-21swapper/221:56:412
29926991815,2cyclictest0-21swapper/100:26:071
2992699180,2cyclictest0-21swapper/121:10:001
2992399182,13cyclictest15649-21fschecks_count19:45:160
2992399180,2cyclictest0-21swapper/023:26:140
2993399173,11cyclictest16072-21/usr/sbin/munin23:20:243
2993399170,1cyclictest0-21swapper/300:14:073
2993099172,1cyclictest141rcu_preempt22:07:082
2993099170,1cyclictest0-21swapper/222:51:462
2993099170,17cyclictest0-21swapper/222:11:372
2992699173,13cyclictest0-21swapper/100:20:151
2992399170,1cyclictest17065-21sort20:55:170
2992399170,16cyclictest0-21swapper/021:52:400
2992399170,16cyclictest0-21swapper/021:52:400
2993399168,2cyclictest1270-21munin-run21:55:003
2993399168,2cyclictest1270-21munin-run21:55:003
2993399164,6cyclictest6069-21apt-get23:15:123
2993399160,7cyclictest0-21swapper/323:03:153
2993099162,13cyclictest0-21swapper/220:10:132
2992699162,3cyclictest0-21swapper/122:10:591
29926991615,1cyclictest25-21ksoftirqd/100:14:411
29926991615,1cyclictest23292-21diskmemload21:46:101
2992699160,13cyclictest0-21swapper/122:38:201
2992399169,6cyclictest10835-21wc20:40:250
2992399161,1cyclictest4909-21cat19:25:000
2992399160,2cyclictest0-21swapper/022:36:340
2992399160,2cyclictest0-21swapper/021:00:240
2992399160,1cyclictest0-21swapper/020:50:150
29933991514,1cyclictest0-21swapper/321:34:333
2993399150,3cyclictest0-21swapper/322:34:043
2993399150,1cyclictest0-21swapper/323:31:103
2993399150,15cyclictest0-21swapper/300:39:443
2993099150,1cyclictest0-21swapper/223:57:552
2993099150,15cyclictest0-21swapper/223:33:392
2992699150,6cyclictest0-21swapper/121:05:141
2992699150,1cyclictest0-21swapper/123:30:121
2992399157,7cyclictest13279-21grep20:45:250
2992399155,5cyclictest2486-21perf19:20:000
2992399150,4cyclictest0-21swapper/019:10:100
2992399150,1cyclictest0-21swapper/020:35:210
2992399150,1cyclictest0-21swapper/020:25:140
2992399150,14cyclictest0-21swapper/019:55:130
2992399150,14cyclictest0-21swapper/019:55:130
2993399147,5cyclictest27352-21df_abs22:10:153
2993399146,5cyclictest29867-21apache200:02:503
29933991410,3cyclictest9607-21awk22:00:003
29933991410,3cyclictest9607-21awk22:00:003
29933991410,3cyclictest1525-21ssh23:50:253
29930991411,2cyclictest2947-21cat00:10:252
2993099140,3cyclictest0-21swapper/200:01:042
2993099140,1cyclictest0-21swapper/222:37:002
2993099140,13cyclictest24793-21cat21:10:182
2992699149,2cyclictest6297-21tr22:55:201
2992699141,2cyclictest141rcu_preempt00:19:111
2992699140,13cyclictest5565-21cron23:15:001
2992399148,5cyclictest4799-21tr20:30:140
2992399148,5cyclictest26655-21taskset20:07:150
2992399148,5cyclictest24297-21taskset20:02:270
2992399148,5cyclictest24297-21taskset20:02:270
2992399148,5cyclictest13383-21grep19:40:150
2992399147,6cyclictest8608-21grep19:30:180
2992399140,2cyclictest523-21head20:20:140
2993399139,3cyclictest8361-21sort19:30:163
2993399139,3cyclictest20460-21sed21:00:253
2993399139,3cyclictest19034-21grep19:50:253
2993399139,3cyclictest11521-21tr22:20:143
2993399138,3cyclictest10733-21ssh22:00:153
2993399132,6cyclictest0-21swapper/300:17:353
29933991310,2cyclictest9968-21sed23:55:263
29933991310,2cyclictest6669-21wc20:35:013
29933991310,2cyclictest3443-21grep00:30:193
29933991310,2cyclictest2431-21tr20:25:123
29933991310,2cyclictest18170-21grep00:20:143
29933991310,2cyclictest13501-21head22:40:193
29933991310,2cyclictest1175-21ls21:15:223
2993399130,10cyclictest0-21swapper/300:25:423
2993099139,3cyclictest7289-21grep23:15:182
2993099139,3cyclictest19042-21sensors21:45:222
2993099139,3cyclictest14923-21cat23:40:002
2993099139,3cyclictest14267-21grep20:50:132
2993099133,9cyclictest7493-21ssh23:54:552
29930991311,2cyclictest32-21ksoftirqd/200:23:502
29930991310,2cyclictest31211-21chrt21:53:042
29930991310,2cyclictest31211-21chrt21:53:042
29930991310,2cyclictest22180-21awk23:25:002
29930991310,2cyclictest22055-21ls22:45:212
2993099130,13cyclictest0-21swapper/223:42:442
2993099130,12cyclictest7218-21perf19:30:002
2993099130,12cyclictest0-21swapper/222:00:012
2992699135,6cyclictest2151-21/usr/sbin/munin00:30:261
2992699135,4cyclictest225-21systemd-journal22:06:281
2992699134,6cyclictest6682-21diskstats23:15:151
2992699131,1cyclictest1047-21kworker/1:223:20:021
29926991310,2cyclictest8282-21grep23:35:191
29926991310,2cyclictest24298-21ls00:05:001
2992699130,1cyclictest0-21swapper/122:02:541
2992699130,13cyclictest0-21swapper/122:21:571
2992399134,8cyclictest28922-21taskset20:10:500
2992399130,2cyclictest5395-21cstates19:25:140
2993399129,2cyclictest30063-21tr20:15:133
2993399129,2cyclictest28848-21sed20:10:253
2993399129,2cyclictest21687-21sed22:25:263
2993399129,2cyclictest19222-21sed22:05:163
2993399129,2cyclictest16941-21grep21:25:123
2993399129,2cyclictest14214-21chrt21:42:513
2993399128,3cyclictest24195-21find23:25:223
2993399120,4cyclictest0-21swapper/320:50:123
2993399120,2cyclictest0-21swapper/323:40:253
2993399120,11cyclictest15795-21grep19:45:163
2993099129,2cyclictest3666-21sort20:25:232
2993099129,2cyclictest23696-21sed23:05:262
2993099129,2cyclictest19583-21grep21:00:162
2993099129,2cyclictest19552-21cat19:55:002
2993099129,2cyclictest10858-21cut22:20:012
2993099129,2cyclictest10836-21ssh00:15:212
2993099129,2cyclictest10128-21grep19:35:132
2993099128,3cyclictest6040-21ls20:30:262
2993099126,2cyclictest225-21systemd-journal20:50:002
2993099120,9cyclictest0-21swapper/219:30:122
2993099120,4cyclictest225-21systemd-journal20:15:252
2993099120,11cyclictest12813-21df22:40:142
2992699129,2cyclictest9529-21grep23:55:211
2992699129,2cyclictest8238-21cut21:20:131
2992699129,2cyclictest7822-21cut19:30:131
2992699129,2cyclictest6295-21ls19:25:211
2992699129,2cyclictest29367-21grep22:50:141
2992699129,2cyclictest28754-21grep20:10:251
2992699129,2cyclictest18091-21sort19:50:161
2992699129,2cyclictest16360-21basename21:25:081
2992699129,2cyclictest15689-21grep19:45:161
2992699129,2cyclictest11016-21grep21:40:261
2992699129,1cyclictest0-21swapper/120:35:121
2992699128,3cyclictest10768-21grep00:35:131
2992699128,3cyclictest0-21swapper/100:05:581
2992699127,4cyclictest24757-21ssh22:47:031
2992699121,9cyclictest23292-21diskmemload23:03:041
29926991210,2cyclictest25-21ksoftirqd/121:38:301
2992699120,12cyclictest0-21swapper/122:30:591
2992699120,11cyclictest15167-21ntpq20:50:181
2992399121,2cyclictest17407-21apache_accesses19:50:110
2993399118,2cyclictest7752-21sort23:35:163
2993399118,2cyclictest750-21ssh22:52:033
2993399118,2cyclictest7191-21ssh22:55:333
2993399118,2cyclictest6084-21ssh22:35:333
2993399118,2cyclictest4833-21ssh22:15:303
2993399118,2cyclictest382-21cut20:20:143
2993399118,2cyclictest354-21ssh23:10:573
2993399118,2cyclictest31623-21ssh00:09:043
2993399118,2cyclictest30915-21head19:10:153
2993399118,2cyclictest25657-21sed23:45:253
2993399118,2cyclictest24960-21egrep20:05:143
2993399118,2cyclictest23085-21tune2fs20:00:163
2993399118,2cyclictest23085-21tune2fs20:00:163
2993399118,2cyclictest22928-21rm22:45:373
2993399118,2cyclictest2181-21ls21:35:233
2993399118,2cyclictest20749-21expr19:55:193
2993399118,2cyclictest20749-21expr19:55:193
2993399118,2cyclictest13470-21hostname19:40:153
2993399118,2cyclictest12429-21grep20:45:163
2993399118,2cyclictest10160-21ssh21:20:293
2993399117,3cyclictest10286-21grep19:35:143
2993399111,5cyclictest225-21systemd-journal20:40:003
29933991110,1cyclictest0-21swapper/323:08:413
2993099119,1cyclictest7236-21expr20:35:132
2993099118,2cyclictest7725-21rm21:39:192
2993099118,2cyclictest32440-21sh23:10:392
2993099118,2cyclictest32239-21taskset19:14:472
2993099118,2cyclictest26961-21grep00:25:162
2993099118,2cyclictest23143-21head20:00:162
2993099118,2cyclictest23143-21head20:00:162
2993099118,2cyclictest21632-21expr21:05:152
2993099118,2cyclictest17262-21grep21:25:142
2993099118,2cyclictest16942-21seq19:47:282
2993099118,2cyclictest15555-21ssh23:00:342
2993099118,2cyclictest12900-21grep19:40:122
2993099118,2cyclictest11911-21tr00:35:202
2993099118,2cyclictest10000-21grep20:40:162
2993099117,3cyclictest5341-21grep22:55:142
2993099117,3cyclictest12186-21grep22:20:172
2993099110,1cyclictest0-21swapper/220:20:112
2992699119,1cyclictest21232-21wc19:55:251
2992699119,1cyclictest21232-21wc19:55:251
2992699118,2cyclictest9580-21cron22:00:001
2992699118,2cyclictest9580-21cron22:00:001
2992699118,2cyclictest9537-21tr20:40:141
2992699118,2cyclictest544-21cut21:15:171
2992699118,2cyclictest4988-21expr20:30:141
2992699118,2cyclictest25607-21basename20:05:191
2992699118,2cyclictest1547-21ls19:15:211
2992699118,2cyclictest11319-21wc19:35:221
2992699117,3cyclictest17896-21cat20:55:251
2992699117,3cyclictest14193-21tail22:40:251
2992699111,2cyclictest225-21systemd-journal21:05:001
2993399108,1cyclictest7113-21taskset19:28:563
2993399108,1cyclictest3616-21cat19:20:173
2993399108,1cyclictest22310-21awk21:05:203
2993399108,1cyclictest17618-21cut20:55:213
2993399107,2cyclictest370-21basename19:15:133
2993399107,2cyclictest11418-21awk20:45:003
2993099108,1cyclictest3045-21expr19:20:142
2993099108,1cyclictest21725-21seq19:57:592
2993099108,1cyclictest21725-21seq19:57:582
2993099107,2cyclictest26877-21sh20:10:002
2993099107,2cyclictest1099-21sed19:15:162
2992699108,1cyclictest40252sleep120:27:041
2992699108,1cyclictest32037-21chrt19:12:111
2992699108,1cyclictest316322sleep120:16:301
2992699108,1cyclictest1339-21sed20:20:231
2992699107,2cyclictest3068-21expr19:20:131
2992699107,2cyclictest24853-21aten2_r7power_e21:30:131
2992699107,2cyclictest24481-21grep23:45:151
2992699107,2cyclictest24011-21grep20:00:251
2992699107,2cyclictest24011-21grep20:00:251
2992699107,2cyclictest14589-21taskset19:42:431
2992699107,2cyclictest11958-21tr20:45:131
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional