You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-31 - 04:48

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack7slot7s.osadl.org (updated Wed Dec 31, 2025 00:43:47)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1315729380,9sleep00-21swapper/019:07:590
1322129076,10sleep20-21swapper/219:08:492
1299428761,12sleep30-21swapper/319:05:523
1117727767,6sleep10-21swapper/119:05:031
1339999262,23cyclictest19832-21wc22:45:001
1340499200,19cyclictest0-21swapper/222:05:152
1339999205,13cyclictest0-21swapper/120:40:141
13399992019,1cyclictest0-21swapper/123:27:291
1340499170,14cyclictest0-21swapper/220:20:132
1339999160,14cyclictest0-21swapper/121:10:181
1339599165,6cyclictest8636-21awk21:14:590
1340699151,10cyclictest0-21swapper/320:50:153
1339999152,9cyclictest22008-21apt-get20:35:141
13399991510,4cyclictest8901-21cut00:35:181
13399991510,3cyclictest18893-21cut23:50:131
1339599159,4cyclictest11194-21cron21:20:000
13395991511,3cyclictest4384-21wc19:55:290
13406991410,3cyclictest4887-21wc00:25:283
13406991410,3cyclictest29388-21head00:10:173
13406991410,3cyclictest27271-21sed19:35:263
1340499149,4cyclictest17911-21grep21:30:282
13404991410,3cyclictest9744-21sed00:35:272
13404991410,3cyclictest10470-21ls23:30:222
1339999149,3cyclictest23483-21tr21:45:161
1339599141,6cyclictest5464-21munin-run22:15:010
13395991410,3cyclictest31630-21grep20:55:150
13395991410,2cyclictest29433-21wc21:55:210
1339599140,7cyclictest0-21swapper/000:05:140
1339599140,13cyclictest22930-21tr22:50:150
1340699139,3cyclictest23462-21grep19:30:163
1340699139,3cyclictest12217-21head21:20:163
1340699139,3cyclictest11896-21grep23:35:163
13406991310,2cyclictest9770-21grep00:35:273
13406991310,2cyclictest3793-21grep23:15:263
13406991310,2cyclictest28638-21grep21:55:153
13406991310,2cyclictest25658-21sort22:55:193
13406991310,2cyclictest18778-21cat23:50:013
13406991310,2cyclictest18778-21cat23:50:013
1340499139,3cyclictest6518-21head00:30:202
1340499139,3cyclictest23873-21ls22:50:222
1340499139,1cyclictest0-21swapper/200:25:162
1340499133,6cyclictest14546-21apache221:35:142
13404991310,2cyclictest4353-21grep21:05:152
13404991310,2cyclictest30379-21ls20:50:232
13404991310,2cyclictest27895-21kill00:06:042
13404991310,2cyclictest14231-21grep19:10:172
13404991310,2cyclictest1242-21grep19:50:182
1340499130,3cyclictest0-21swapper/221:40:142
1340499130,12cyclictest31901-21cron23:10:002
1340499130,12cyclictest29213-21awk19:40:252
1340499130,12cyclictest23328-21cron00:00:012
1339999139,2cyclictest3809-21cat21:05:001
13399991310,2cyclictest9115-21wc20:05:251
13399991310,2cyclictest4962-21grep22:10:291
13399991310,2cyclictest27667-21tr23:00:161
13399991310,2cyclictest27087-21wc21:50:191
13399991310,2cyclictest27087-21wc21:50:191
13399991310,2cyclictest24827-21grep19:30:271
13399991310,2cyclictest1645-21tune2fs00:20:191
13399991310,2cyclictest13131-21wc23:35:271
13399991310,2cyclictest13122-21tr22:30:141
13399991310,2cyclictest11379-21cron23:34:591
1339599139,3cyclictest24550-21grep21:45:220
1339599139,2cyclictest20729-21grep20:30:200
1339599139,2cyclictest18174-21cron19:20:000
1339599138,4cyclictest3613-21wc23:15:260
1339599136,5cyclictest30769-21/usr/sbin/munin22:00:270
1339599132,5cyclictest1886-21perf23:15:000
1339599132,4cyclictest0-21swapper/021:30:150
13395991310,2cyclictest25314-21cron19:35:000
13395991310,2cyclictest25070-21wc00:00:280
13395991310,2cyclictest20113-21wc23:50:230
13395991310,2cyclictest17905-21sort20:25:170
13395991310,2cyclictest14035-21grep20:15:270
13395991310,2cyclictest10292-21cat22:25:010
13395991310,1cyclictest7155-21grep23:25:150
1340699129,2cyclictest32317-21cut20:55:193
1340699129,2cyclictest28739-21grep19:40:203
1340699129,2cyclictest28155-21head23:00:173
1340699129,2cyclictest27099-21grep00:05:223
1340699129,2cyclictest26780-21head21:50:173
1340699129,2cyclictest26780-21head21:50:173
1340699129,2cyclictest21786-21head21:40:173
1340699129,2cyclictest2092-21grep00:20:223
1340699129,2cyclictest19953-21cat23:50:213
1340699129,2cyclictest1856-21cron23:15:003
1340699129,2cyclictest16375-21tr21:30:163
1340699129,2cyclictest15337-21grep22:35:123
1340699129,2cyclictest10773-21grep22:25:153
1340699128,3cyclictest24225-21cat22:50:273
1340699128,2cyclictest8419-21sed20:05:173
1340699128,2cyclictest13926-21cat22:30:193
1340699126,2cyclictest225-21systemd-journal20:50:003
1340499129,2cyclictest8217-21sed21:10:272
1340499129,2cyclictest3259-21tr23:15:192
1340499129,2cyclictest28902-21tr00:10:152
1340499129,2cyclictest26083-21sed20:40:272
1340499129,2cyclictest25136-21grep22:55:162
1340499129,2cyclictest18108-21grep23:45:272
1340499129,2cyclictest18108-21grep23:45:272
1340499129,2cyclictest14617-21perf20:20:012
1340499129,2cyclictest11788-21grep23:35:162
1340499128,2cyclictest18346-21grep22:40:182
1339999129,2cyclictest4086-21head00:25:191
1339999129,2cyclictest29129-21grep21:55:181
1339999129,2cyclictest28915-21tr00:10:151
1339999129,2cyclictest27705-21munin-plugin-st19:39:591
1339999129,2cyclictest25807-21grep22:55:191
1339999129,2cyclictest22693-21grep22:50:141
1339999129,2cyclictest21046-21grep20:30:271
1339999129,2cyclictest18441-21wc21:35:001
1339999129,2cyclictest15627-21grep21:25:271
1339999129,2cyclictest12143-21grep22:25:261
1339999128,3cyclictest2860-21sh22:10:011
1339999128,3cyclictest1631-21head19:50:211
1339999127,4cyclictest4235-21run-parts19:55:251
1339999127,2cyclictest18323-21taskset23:46:431
1339999127,2cyclictest18323-21taskset23:46:431
1339999126,2cyclictest225-21systemd-journal20:15:011
13399991210,1cyclictest26767-21grep00:05:181
1339599129,2cyclictest3480-21grep00:25:160
1339599129,2cyclictest30084-21cron19:45:000
1339599129,2cyclictest27709-21munin-run19:40:000
1339599129,2cyclictest25320-21grep22:55:170
1339599129,2cyclictest18957-21wc22:40:230
1339599129,2cyclictest13886-21tr19:10:140
1339599128,3cyclictest10744-21tr22:25:140
1339599128,2cyclictest19400-21grep21:35:170
1339599128,2cyclictest1896-21tr22:05:220
1339599128,2cyclictest13934-21wc23:40:010
1339599120,11cyclictest28132-21grep23:00:180
1340699119,1cyclictest9925-21sort23:30:163
1340699119,1cyclictest5282-21grep23:20:163
1340699118,2cyclictest8791-21grep22:20:163
1340699118,2cyclictest6461-21tail00:30:213
1340699118,2cyclictest3648-21sh21:05:003
1340699118,2cyclictest2988-21apt-config22:10:013
1340699118,2cyclictest24411-21cut20:40:143
1340699118,2cyclictest24233-21cut00:00:193
1340699118,2cyclictest22723-21cat23:55:283
1340699118,2cyclictest21523-21seq20:32:253
1340699118,2cyclictest19705-21wc21:35:203
1340699118,2cyclictest17923-21cut20:25:163
1340699118,2cyclictest13898-21cut21:25:153
1340699118,2cyclictest10711-21tune2fs20:10:173
1340699117,3cyclictest7395-21cron20:04:593
1340699117,3cyclictest10710-21grep21:15:263
1340699114,3cyclictest225-21systemd-journal22:15:013
1340699113,4cyclictest225-21systemd-journal19:20:003
1340699110,10cyclictest23489-21aten2_r7power_e21:45:153
1340499118,2cyclictest8321-21sort20:05:162
1340499118,2cyclictest5895-21sort20:00:162
1340499118,2cyclictest31895-21cat22:00:212
1340499118,2cyclictest31171-21sort19:45:182
1340499118,2cyclictest27745-21expr23:00:152
1340499118,2cyclictest27692-21sh19:39:592
1340499118,2cyclictest25574-21chrt00:03:072
1340499118,2cyclictest21631-21taskset20:33:482
1340499118,2cyclictest21210-21ntpq22:45:212
1340499118,2cyclictest16740-21fschecks_time19:15:162
1340499118,2cyclictest14156-21grep23:40:152
1340499118,2cyclictest1149-21taskset20:58:582
1340499118,2cyclictest11210-21idleruntime-cro21:20:002
1340499114,5cyclictest25986-21apt-get21:50:032
1340499114,5cyclictest25986-21apt-get21:50:032
1340499112,7cyclictest449-21irqbalance21:00:122
1340499110,10cyclictest6671-21latency22:15:202
1340499110,10cyclictest32727-21sed00:15:282
1339999118,2cyclictest7409-21grep22:15:271
1339999118,2cyclictest6342-21chrt23:21:231
1339999118,2cyclictest6244-21grep21:10:001
1339999118,2cyclictest32488-21chrt19:47:421
1339999118,2cyclictest32482-21head22:00:281
1339999118,2cyclictest31791-21taskset23:08:361
1339999118,2cyclictest29920-21chrt19:42:571
1339999118,2cyclictest27395-21expr20:45:191
1339999118,2cyclictest25800-21cat00:05:011
1339999118,2cyclictest22512-21tail19:25:261
1339999118,2cyclictest21984-21tail23:55:191
1339999117,1cyclictest4011-21chrt23:17:091
1339999116,4cyclictest14244-21taskset20:16:271
1339999111,9cyclictest15881-21diskstats22:35:151
1339599119,1cyclictest27586-21wc20:45:210
1339599118,2cyclictest8218-21grep20:05:150
1339599118,2cyclictest7104-21chrt20:01:200
1339599118,2cyclictest5253-21ntpq21:05:210
1339599118,2cyclictest3457-21chrt21:02:420
1339599118,2cyclictest29290-21cut20:50:160
1339599118,2cyclictest24504-21cut20:40:150
1339599118,2cyclictest21988-21grep23:55:190
1339599118,2cyclictest19226-21tail19:20:190
1339599118,2cyclictest16732-21tr23:45:160
1339599118,2cyclictest16732-21tr23:45:150
1339599118,2cyclictest16074-21head22:35:160
1339599118,2cyclictest15078-21sed23:40:200
1339599118,2cyclictest1254-21tr00:20:150
1339599118,2cyclictest11745-21head20:10:270
1339599118,2cyclictest10418-21awk23:30:200
1339599117,3cyclictest7323-21wc00:30:300
1339599113,5cyclictest16985-21cron20:25:010
1340699108,1cyclictest8044-21ntpq23:25:213
1340699108,1cyclictest7643-21ntpq21:10:223
1340699108,1cyclictest6161-21ls22:15:163
1340699108,1cyclictest4750-21chrt19:56:503
1340699108,1cyclictest32382-21cat22:00:273
1340699108,1cyclictest31511-21grep23:05:283
1340699108,1cyclictest30983-21chrt00:15:053
1340699108,1cyclictest22434-21expr20:35:153
1340699108,1cyclictest20435-21expr22:45:163
1340699108,1cyclictest17911-21cpuspeed_turbos22:40:163
1340699108,1cyclictest15826-21cut20:20:183
1340699108,1cyclictest14245-21cut19:10:173
1340699108,1cyclictest13178-21expr20:15:183
1340699107,2cyclictest30792-21tr19:45:153
1340699107,2cyclictest14012-21taskset23:40:043
1340699103,3cyclictest20845-21apt-get19:25:133
1340699100,9cyclictest3998-21gpgv21:05:003
1340699100,9cyclictest18576-21grep19:20:163
1340499108,1cyclictest8150-21wget22:20:142
1340499108,1cyclictest3548-21fschecks_time19:55:202
1340499108,1cyclictest26917-21cpu20:45:172
1340499108,1cyclictest20816-21basename19:25:122
1340499108,1cyclictest1613-21seq23:11:562
1340499108,1cyclictest15768-21cat22:35:132
1340499108,1cyclictest13579-21head22:30:172
1340499107,2cyclictest3124-21basename00:25:002
1340499107,2cyclictest29717-21grep21:55:222
1340499107,2cyclictest24152-21grep21:45:202
1340499107,2cyclictest22286-21tr20:35:152
1340499107,2cyclictest19499-21tune2fs23:50:162
1340499107,2cyclictest15430-21timerandwakeup21:25:272
1340499107,2cyclictest12347-21cat21:20:172
1340499107,2cyclictest11903-21wc22:25:262
1340499103,3cyclictest3394-21aten2_r7power_e22:10:162
1340499103,2cyclictest0-21swapper/220:25:132
1340499100,9cyclictest8676-21taskset23:25:502
1340499100,9cyclictest18940-21expr19:20:162
1339999108,1cyclictest32299-21grep23:10:141
1339999108,1cyclictest31184-21taskset20:53:441
1339999108,1cyclictest14110-21basename23:40:151
1339999108,1cyclictest13145-21sed21:20:251
1339999107,2cyclictest9816-21tail21:15:171
1339999107,2cyclictest6050-21ls00:30:181
1339999107,2cyclictest348-21wc20:55:231
1339999107,2cyclictest31768-21grep00:15:191
1339999107,2cyclictest22200-21grep21:40:211
1339999107,2cyclictest20721-21sort22:45:181
1339999107,2cyclictest19064-21expr21:35:151
1339999107,2cyclictest18428-21chrt19:20:051
1339999107,2cyclictest17802-21df20:25:161
1339999106,3cyclictest15600-21grep20:20:161
1339999106,3cyclictest15010-21grep19:10:231
1339999105,4cyclictest18150-21taskset19:19:461
1339999101,2cyclictest225-21systemd-journal22:22:331
1339599108,1cyclictest75362sleep022:16:270
1339599108,1cyclictest5888-21awk23:20:210
1339599108,1cyclictest31711-21taskset23:07:330
1339599108,1cyclictest28751-21wget00:10:150
1339599108,1cyclictest22971-21taskset21:42:310
1339599108,1cyclictest147062sleep022:30:530
1339599107,2cyclictest31310-21cut00:15:170
1339599107,2cyclictest28061-21chrt21:54:110
1339599107,2cyclictest28061-21chrt21:54:100
1339599107,2cyclictest24135-21awk20:40:000
1339599107,2cyclictest2335-21taskset19:51:200
1339599107,2cyclictest14129-21tr21:25:160
1339599100,9cyclictest9119-21cat00:35:200
1339599100,9cyclictest13301-21taskset21:21:220
134069997,1cyclictest702-21egrep19:50:153
134049997,1cyclictest4749-21sort23:20:142
134049997,1cyclictest23235-21cut19:30:152
134049997,1cyclictest10503-21expr20:10:152
133959997,1cyclictest32584-21chrt19:48:570
133959997,1cyclictest22703-21chrt19:26:580
133959996,2cyclictest20808-21grep22:45:180
133999981,6cyclictest5920-21fschecks_count20:00:171
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional