You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-03 - 06:07

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot7s.osadl.org (updated Mon Aug 03, 2026 00:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1747929582,9sleep20-21swapper/219:07:312
1739029178,9sleep00-21swapper/019:06:200
1749328665,6sleep10-21swapper/119:07:421
1736328370,9sleep30-21swapper/319:05:593
47822670,1sleep2311rcuc/222:04:252
1776199290,0cyclictest0-21swapper/222:32:102
1776199280,28cyclictest0-21swapper/221:29:242
1776199270,12cyclictest0-21swapper/222:35:032
1776199264,2cyclictest10549-21diskmemload23:50:152
1776199264,2cyclictest10549-21diskmemload23:50:152
1776199260,2cyclictest0-21swapper/200:23:182
1776199260,24cyclictest0-21swapper/223:55:202
1776199260,23cyclictest0-21swapper/222:49:502
1776199260,1cyclictest0-21swapper/200:05:162
1776199260,15cyclictest0-21swapper/221:13:462
17756992622,3cyclictest31004-21tr00:15:161
1776199250,6cyclictest0-21swapper/221:55:152
1776199250,1cyclictest0-21swapper/200:36:362
1775299240,22cyclictest0-21swapper/022:07:480
17763992317,1cyclictest30975-21munin-run22:20:003
1776199230,23cyclictest0-21swapper/223:47:502
1775699230,0cyclictest0-21swapper/123:48:051
1775299236,7cyclictest0-21swapper/019:15:140
1776199221,1cyclictest0-21swapper/222:07:402
1775699220,21cyclictest0-21swapper/123:57:441
1775699220,21cyclictest0-21swapper/119:40:111
1776399215,9cyclictest8702-21/usr/sbin/munin22:45:113
1776199210,0cyclictest0-21swapper/221:47:202
1775699210,1cyclictest0-21swapper/123:33:031
1776399206,2cyclictest141rcu_preempt23:21:503
17761992010,2cyclictest0-21swapper/220:05:212
1776199200,1cyclictest0-21swapper/223:27:052
17756992016,4cyclictest0-21swapper/123:39:261
17763991912,4cyclictest445-21cut22:40:123
1776199190,16cyclictest0-21swapper/222:18:012
1775699190,18cyclictest0-21swapper/100:34:031
1775299194,8cyclictest12245-21/usr/sbin/munin23:45:130
110672190,0sleep30-21swapper/321:10:123
1776399184,2cyclictest141rcu_preempt23:25:313
1776199182,2cyclictest0-21swapper/222:40:132
1776199180,1cyclictest0-21swapper/221:38:062
1776199180,0cyclictest0-21swapper/222:24:362
1775299186,6cyclictest23750-21perf22:35:010
17752991810,7cyclictest1370-21apache221:29:280
1776199179,2cyclictest7378-21wc21:00:232
1776199179,2cyclictest28547-21grep19:30:182
1776199179,2cyclictest27537-21wc21:20:002
1776199179,2cyclictest25396-21rm22:55:042
17761991715,1cyclictest0-21swapper/223:18:552
1776199170,3cyclictest0-21swapper/223:20:142
1775699170,17cyclictest0-21swapper/121:27:291
1775699170,16cyclictest0-21swapper/121:46:491
1775299170,0cyclictest0-21swapper/021:19:230
1776399162,1cyclictest0-21swapper/300:25:143
1776399160,1cyclictest0-21swapper/300:23:213
1776399160,14cyclictest0-21swapper/323:12:023
1776199169,2cyclictest8541-21tr21:05:152
1776199168,2cyclictest30657-21sed20:40:262
1776199168,2cyclictest2789-21ssh23:00:202
1776199168,2cyclictest19811-21grep23:30:142
17761991610,2cyclictest11195-21ls20:00:282
17756991615,1cyclictest0-21swapper/123:04:551
17756991615,1cyclictest0-21swapper/122:53:481
1775699160,16cyclictest0-21swapper/122:24:171
1775699160,15cyclictest0-21swapper/123:12:201
1775299160,4cyclictest0-21swapper/022:26:250
17763991511,2cyclictest30041-21cat19:35:003
17763991511,2cyclictest11110-21wc23:05:193
1776199159,1cyclictest19630-21ntpq23:10:212
1776199157,2cyclictest31411-21grep19:35:202
1776199154,3cyclictest225-21systemd-journal19:20:002
1776199150,2cyclictest0-21swapper/200:15:442
1776199150,1cyclictest0-21swapper/200:04:232
1775699154,5cyclictest241rcuc/100:10:121
17756991514,1cyclictest0-21swapper/122:01:141
1775699150,14cyclictest0-21swapper/100:25:131
1775299153,6cyclictest21517-21awk20:25:000
1775299152,7cyclictest25087-21munin-run22:55:000
17752991512,2cyclictest0-21swapper/022:11:580
1775299150,13cyclictest0-21swapper/021:50:550
17763991411,2cyclictest32630-21ssh00:15:283
17763991410,2cyclictest4215-21tr23:40:143
1776399140,10cyclictest0-21swapper/323:55:123
1776199144,6cyclictest537-21snmpd22:51:032
1776199143,1cyclictest21694-21ssh00:29:052
1776199142,10cyclictest1142-21runrttasks23:08:192
17761991412,2cyclictest0-21swapper/221:44:152
17761991410,2cyclictest310-21grep19:40:122
1776199140,1cyclictest0-21swapper/223:44:242
1775699141,8cyclictest10549-21diskmemload00:00:141
17756991411,2cyclictest22578-21wc22:15:011
1775699140,13cyclictest0-21swapper/121:01:231
1775299146,7cyclictest4547-21sendmail00:00:000
1775299145,3cyclictest3403-21munin-plugin-st20:55:000
17752991411,2cyclictest3420-21awk23:40:000
17752991410,2cyclictest5030-21grep21:45:140
17752991410,2cyclictest29286-21tr21:40:140
1775299140,13cyclictest0-21swapper/023:22:470
1776399139,3cyclictest9349-21cron20:00:003
1776399139,3cyclictest5751-21grep00:00:173
1776399139,3cyclictest28591-21sh23:55:003
1776399139,3cyclictest28591-21sh23:55:003
1776399136,5cyclictest23790-21cut22:35:013
1776399131,10cyclictest14686-21tr21:50:223
17763991310,2cyclictest9156-21wc22:25:273
17763991310,2cyclictest25925-21grep19:25:183
17763991310,2cyclictest25896-21head20:30:273
17763991310,2cyclictest22519-21apt-config22:15:013
17763991310,2cyclictest19531-21wc19:10:273
1776199139,3cyclictest26299-21cron20:35:002
1776199139,2cyclictest3936-21grep20:55:162
17761991310,2cyclictest23585-21grep19:20:172
1776199130,0cyclictest0-21swapper/223:37:562
17756991311,2cyclictest10549-21diskmemload21:32:021
1775699131,11cyclictest25678-21basename22:55:161
17756991310,2cyclictest29727-21wc21:20:261
17756991310,2cyclictest28604-21cat21:40:001
17756991310,2cyclictest25-21ksoftirqd/120:40:011
17756991310,2cyclictest16650-21awk22:50:001
17756991310,2cyclictest14287-21dpkg20:10:001
17756991310,2cyclictest0-21swapper/123:40:191
1775299139,3cyclictest23604-21df22:15:150
1775299139,3cyclictest21345-21wc23:30:250
1775299139,2cyclictest21477-21grep23:50:180
1775299139,2cyclictest21477-21grep23:50:180
1775299135,6cyclictest6968-21runrttasks00:39:470
1775299135,2cyclictest18938-21idleruntime-cro21:15:000
1775299132,6cyclictest28698-21munin-run20:40:000
17752991311,2cyclictest0-21swapper/000:15:130
17752991310,2cyclictest24535-21head19:20:260
17752991310,2cyclictest23681-21gzip00:10:220
17752991310,2cyclictest14148-21sed21:30:270
17752991310,2cyclictest11737-21df23:25:150
1776399129,2cyclictest5390-21chrt22:04:543
1776399129,2cyclictest32307-21grep22:20:173
1776399129,2cyclictest32036-21cut20:45:183
1776399129,2cyclictest30575-21sed20:40:263
1776399129,2cyclictest27036-21grep22:55:243
1776399129,2cyclictest12584-21tr23:45:153
1776399128,3cyclictest13576-21tr00:05:163
1776399128,2cyclictest22534-21sh19:20:003
1776399128,2cyclictest17217-21tr20:15:143
1776399128,2cyclictest10327-21sort20:00:183
1776399127,4cyclictest28139-21tr23:35:133
1776399127,3cyclictest20960-21wc20:20:253
1776399124,4cyclictest225-21systemd-journal00:10:253
1776399123,8cyclictest8011-21unixbench_singl22:05:273
1776399123,5cyclictest2213-21latency_hist23:20:003
1776399122,6cyclictest0-21swapper/321:20:113
1776399122,6cyclictest0-21swapper/300:30:013
1776399121,10cyclictest8843-21sh21:27:473
1776399120,0cyclictest0-21swapper/300:37:223
1776199121,10cyclictest198-21jbd2/sda2-800:30:012
1775699129,3cyclictest0-21swapper/122:25:171
1775699129,2cyclictest31873-21grep19:35:251
1775699129,2cyclictest30985-21basename22:20:001
1775699129,2cyclictest21838-21uname23:50:201
1775699129,2cyclictest21838-21uname23:50:201
1775699129,2cyclictest1210-21wc20:50:011
1775699127,4cyclictest21028-21ssh21:15:231
1775699126,2cyclictest225-21systemd-journal23:25:001
17756991210,2cyclictest25-21ksoftirqd/120:40:221
17756991210,1cyclictest25-21ksoftirqd/122:42:261
17756991210,1cyclictest0-21swapper/100:05:151
1775699120,11cyclictest12804-21ls23:25:241
1775299129,2cyclictest9601-21grep22:45:160
1775299129,2cyclictest7338-21grep00:20:160
1775299129,2cyclictest32038-21grep22:20:150
1775299129,2cyclictest31084-21ls20:45:000
1775299129,2cyclictest26190-21grep22:55:170
1775299129,2cyclictest25948-21grep19:25:180
1775299129,2cyclictest233952sleep023:12:470
1775299129,2cyclictest1880-21grep23:00:150
1775299129,2cyclictest18093-21grep20:15:190
1775299128,3cyclictest21014-21systemd00:09:410
1775299128,2cyclictest6314-21cut21:00:140
1775299128,2cyclictest5019-21tr19:50:140
1775299123,7cyclictest0-21swapper/000:28:110
1776399118,2cyclictest9448-21load21:05:203
1776399118,2cyclictest7827-21ssh21:46:083
1776399118,2cyclictest5017-21grep19:50:143
1776399118,2cyclictest4423-21cut20:55:193
1776399118,2cyclictest4128-21grep19:45:273
1776399118,2cyclictest29103-21basename21:40:133
1776399118,2cyclictest24614-21grep22:35:143
1776399118,2cyclictest2376-21needreboot20:50:203
1776399118,2cyclictest23313-21df19:20:153
1776399118,2cyclictest23185-21cat20:25:233
1776399118,2cyclictest22858-21sort21:55:213
1776399118,2cyclictest20480-21ssh23:30:173
1776399118,2cyclictest1744-21wget23:00:153
1776399118,2cyclictest17131-21ssh22:50:113
1776399118,2cyclictest15302-21ssh21:31:183
1776399118,2cyclictest13511-21sort20:05:263
1776399117,3cyclictest20246-21grep21:15:173
1776399111,5cyclictest0-21swapper/321:35:563
1776199118,2cyclictest6582-21sed19:50:262
1776199118,2cyclictest31411-21cut20:45:152
1776199118,2cyclictest26019-21ssh00:11:442
1776199118,2cyclictest23941-21latency_hist20:30:002
1776199118,2cyclictest21007-21grep20:20:252
1776199118,2cyclictest1956-21tune2fs20:50:172
1776199118,2cyclictest18269-21cut19:10:152
1776199118,2cyclictest17895-21chrt20:15:182
1776199118,2cyclictest16248-21sed22:10:252
1776199118,2cyclictest16122-21ssh21:31:562
1776199118,2cyclictest13444-21cut21:50:142
1776199114,6cyclictest10549-21diskmemload22:25:032
1776199111,6cyclictest225-21systemd-journal19:49:552
1776199110,10cyclictest9237-21chrt19:58:352
1776199110,10cyclictest27887-21ssh21:20:122
1775699119,1cyclictest18775-21cut19:10:181
1775699118,2cyclictest7936-21ls00:20:201
1775699118,2cyclictest6852-21head22:05:181
1775699118,2cyclictest5427-21taskset20:55:451
1775699118,2cyclictest32618-21grep00:35:181
1775699118,2cyclictest29832-21head21:40:161
1775699118,2cyclictest26308-21cut19:25:211
1775699118,2cyclictest23797-21taskset20:28:261
1775699118,2cyclictest21242-21taskset20:21:511
1775699118,2cyclictest20342-21ssh22:32:251
1775699117,3cyclictest26878-21hostname23:15:141
1775699117,2cyclictest26316-21munin-plugin-st20:35:001
1775699117,1cyclictest23605-21cat21:55:281
1775699110,10cyclictest13500-21chrt23:06:381
1775299119,1cyclictest0-21swapper/022:00:140
1775299118,2cyclictest9036-21sort21:05:170
1775299118,2cyclictest6271-21ssh23:40:410
1775299118,2cyclictest4404-21head20:55:190
1775299118,2cyclictest31853-21head19:35:250
1775299118,2cyclictest29885-21cron19:35:000
1775299118,2cyclictest29742-21ssh23:16:120
1775299118,2cyclictest27451-21ssh00:32:010
1775299118,2cyclictest26402-21ssh21:57:320
1775299118,2cyclictest25310-21expr22:35:170
1775299118,2cyclictest2453-21ssh22:40:280
1775299118,2cyclictest21386-21grep21:35:180
1775299118,2cyclictest14157-21ssh23:07:100
1775299118,2cyclictest12164-21cut20:05:140
1775299118,2cyclictest10696-21ssh00:03:160
1775299111,7cyclictest0-21swapper/019:45:140
1775299110,10cyclictest32084-21ssh21:22:070
1776399108,1cyclictest27288-21head20:35:183
1776399107,2cyclictest845-21tail19:40:143
1776399107,2cyclictest8164-21sh21:05:003
1776399107,2cyclictest14647-21grep20:10:143
1776199108,1cyclictest25178-21cat19:25:122
1776199108,1cyclictest15058-21expr20:10:152
1776199107,2cyclictest26538-21chrt20:35:092
1775699108,1cyclictest9927-21expr20:00:161
1775699108,1cyclictest5443-21sed19:50:151
1775699108,1cyclictest17275-21grep20:15:141
1775699108,1cyclictest13192-21ssh21:50:101
1775699107,2cyclictest8679-21expr21:05:151
1775699107,2cyclictest7463-21cat19:55:161
1775699107,2cyclictest2823-21cat20:50:261
1775699107,2cyclictest20615-21grep19:15:151
1775699107,2cyclictest14811-21sort20:10:141
1775699107,2cyclictest10549-21diskmemload22:35:181
1775699101,5cyclictest0-21swapper/121:10:161
1775699100,6cyclictest225-21systemd-journal19:45:001
1775299108,1cyclictest7701-21chrt19:55:160
1775299108,1cyclictest32403-21cat20:45:210
1775299108,1cyclictest11484-21chrt20:01:370
1775299107,2cyclictest324-21tr19:40:120
1775299107,2cyclictest24237-21grep20:30:150
1775299107,2cyclictest22167-21cat20:25:150
1775299107,2cyclictest19738-21taskset19:10:330
1775299107,2cyclictest16708-21taskset20:14:570
177639997,1cyclictest319192sleep319:35:403
177569997,1cyclictest22939-21basename19:20:141
177569996,2cyclictest27856-21cut19:30:151
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional