You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-29 - 06:25

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot7s.osadl.org (updated Mon Jun 29, 2026 00:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1496229368,9sleep30-21swapper/319:06:373
1508528976,9sleep00-21swapper/019:08:110
1502328766,6sleep20-21swapper/219:07:242
1511828264,14sleep10-21swapper/119:08:361
1536999280,0cyclictest0-21swapper/121:24:091
15376992524,1cyclictest0-21swapper/323:35:103
15376992423,1cyclictest0-21swapper/322:20:513
1537699240,0cyclictest0-21swapper/300:18:103
1537699240,0cyclictest0-21swapper/300:18:103
1536999230,1cyclictest0-21swapper/123:36:411
1536999230,1cyclictest0-21swapper/123:08:231
15376992220,2cyclictest0-21swapper/300:30:163
1537699220,0cyclictest0-21swapper/321:55:063
1536999220,22cyclictest0-21swapper/123:25:471
1537699203,1cyclictest0-21swapper/322:06:323
1537499200,18cyclictest0-21swapper/223:42:192
1537499200,18cyclictest0-21swapper/200:33:252
15369992017,2cyclictest4551-21diskstats22:05:171
1537699192,14cyclictest949-21python23:00:263
15376991918,1cyclictest0-21swapper/321:15:053
15376991917,1cyclictest25591-21turbostat.cron23:35:003
15376991910,3cyclictest8148-21diskmemload00:11:203
1537499190,2cyclictest0-21swapper/222:05:172
148982190,0sleep00-21swapper/023:47:190
1537699189,9cyclictest0-21swapper/321:28:383
1537699187,10cyclictest0-21swapper/323:11:453
15376991817,1cyclictest0-21swapper/300:24:553
15376991810,7cyclictest0-21swapper/322:44:473
1537699180,3cyclictest0-21swapper/323:44:203
1537699180,2cyclictest0-21swapper/323:59:223
1536999189,4cyclictest0-21swapper/100:16:411
1536999189,4cyclictest0-21swapper/100:16:411
1536999180,17cyclictest0-21swapper/123:21:341
1536599185,9cyclictest0-21swapper/019:20:140
1537699179,8cyclictest0-21swapper/321:47:433
1537699172,14cyclictest0-21swapper/321:53:353
15376991715,2cyclictest0-21swapper/323:49:353
15376991715,2cyclictest0-21swapper/300:27:343
15376991712,3cyclictest0-21swapper/323:18:323
1537699170,1cyclictest0-21swapper/321:35:363
1537499173,12cyclictest19947-21/usr/sbin/munin23:50:192
1537499170,1cyclictest0-21swapper/223:45:132
15369991712,4cyclictest30889-21ls00:35:191
1536999170,1cyclictest0-21swapper/121:45:311
1536999170,16cyclictest0-21swapper/123:51:571
1537699168,7cyclictest0-21swapper/323:27:423
15376991614,1cyclictest8148-21diskmemload22:01:483
15376991613,1cyclictest8148-21diskmemload22:33:573
15374991615,1cyclictest0-21swapper/223:38:152
1536999160,1cyclictest0-21swapper/100:22:091
15365991614,1cyclictest8148-21diskmemload23:24:220
15365991612,3cyclictest13-21ksoftirqd/019:25:130
1537699153,10cyclictest20863-21df19:20:163
15376991513,1cyclictest8148-21diskmemload23:50:213
15376991513,1cyclictest0-21swapper/322:25:143
15376991512,2cyclictest8148-21diskmemload22:54:533
15374991512,2cyclictest24264-21wc19:25:252
15374991511,2cyclictest55650irq/127-enp2s0-21:53:552
1537499150,14cyclictest0-21swapper/200:21:452
15369991515,0cyclictest0-21swapper/122:32:521
15369991512,2cyclictest0-21swapper/100:02:121
15369991510,4cyclictest15268-21apache222:03:481
1536599154,3cyclictest14153-21sh22:50:000
1536599153,6cyclictest12830-21ssh22:30:000
1536599151,8cyclictest18890-21/usr/sbin/munin23:50:190
1536599150,1cyclictest0-21swapper/023:06:220
1537699149,5cyclictest0-21swapper/300:06:013
1537699144,8cyclictest0-21swapper/322:35:053
1537699144,8cyclictest0-21swapper/322:35:053
1537699143,10cyclictest23260-21ssh22:15:513
15376991412,1cyclictest8148-21diskmemload21:44:293
15376991412,1cyclictest8148-21diskmemload21:30:233
15376991410,2cyclictest17705-21cron19:15:003
15376991410,1cyclictest8148-21diskmemload21:14:413
15374991411,2cyclictest27891-21wc20:40:212
15374991410,3cyclictest32425-21uname20:50:202
15374991410,3cyclictest32425-21uname20:50:202
15374991410,3cyclictest18302-21wc21:15:212
15374991410,3cyclictest10028-21sed21:10:222
1537499140,1cyclictest0-21swapper/222:30:122
1537499140,13cyclictest28230-21munin-run00:15:012
1537499140,13cyclictest0-21swapper/221:36:192
1536999149,5cyclictest38150irq/131-snd_hda22:20:161
1536999149,3cyclictest18054-21sed20:20:221
1536999145,4cyclictest225-21systemd-journal23:00:001
15369991414,0cyclictest0-21swapper/122:41:391
15369991412,1cyclictest0-21swapper/119:50:191
15369991410,3cyclictest26214-21wc23:15:261
15369991410,3cyclictest21633-21head00:10:221
15369991410,3cyclictest17022-21wc19:10:271
15369991410,2cyclictest2971-21grep20:55:261
1536999140,12cyclictest9541-21cat21:30:001
1536599147,6cyclictest21221-21munin-run22:35:000
1536599144,3cyclictest225-21systemd-journal23:32:320
1536599141,9cyclictest27511-21apt-get19:35:150
1536599141,10cyclictest28715-21apache_accesses00:15:120
1536599141,10cyclictest28715-21apache_accesses00:15:120
1536599140,13cyclictest0-21swapper/021:05:150
1537699138,4cyclictest0-21swapper/322:11:553
1537699135,7cyclictest0-21swapper/320:10:143
1537699134,6cyclictest17591-21grep20:20:173
15376991311,1cyclictest8148-21diskmemload22:49:593
15376991310,2cyclictest4809-21ssh00:00:233
15376991310,2cyclictest32040-21grep20:50:173
15376991310,2cyclictest32040-21grep20:50:173
15376991310,2cyclictest29805-21sort19:40:003
15376991310,2cyclictest1858-21wc23:20:183
1537699130,12cyclictest25399-21grep20:35:223
15374991310,2cyclictest5205-21wc21:00:262
15374991310,2cyclictest29064-21tr00:15:152
15374991310,2cyclictest29064-21tr00:15:152
15374991310,2cyclictest24922-21sed20:35:172
15374991310,2cyclictest21422-21grep22:15:172
15374991310,2cyclictest18766-21cut19:15:162
15374991310,2cyclictest15670-21grep19:10:132
1537499130,2cyclictest0-21swapper/223:04:042
1537499130,12cyclictest0-21swapper/223:07:462
1536999139,3cyclictest26277-21cron20:40:011
1536999139,2cyclictest15781-21wc22:50:181
1536999138,4cyclictest4254-21ssh23:02:441
1536999135,7cyclictest27234-21munin-node21:42:231
1536999131,11cyclictest8148-21diskmemload21:56:461
15369991310,2cyclictest32708-21tr19:45:141
15369991310,2cyclictest23901-21cron20:35:001
15369991310,2cyclictest18607-21tr21:35:151
15369991310,2cyclictest15693-21wc20:15:221
1536999130,13cyclictest0-21swapper/123:11:401
1536599139,3cyclictest29386-21tr22:20:140
1536599139,3cyclictest23460-21grep00:30:270
1536599139,3cyclictest16776-21wc21:15:010
15365991310,2cyclictest9425-21cat20:05:010
15365991310,2cyclictest5421-21tr00:40:000
15365991310,2cyclictest31701-21grep22:40:210
15365991310,2cyclictest16065-21grep22:50:210
1536599130,12cyclictest5843-21cat21:05:000
1537699129,2cyclictest5489-21sort19:55:173
1537699129,2cyclictest30898-21ls00:35:203
1537699129,2cyclictest30837-21grep19:40:163
1537699129,2cyclictest18234-21grep19:15:143
1537699129,2cyclictest10348-21head20:05:173
1537699128,3cyclictest23563-21taskset20:30:443
1537699127,3cyclictest0-21swapper/323:05:283
15376991210,1cyclictest25404-21cut19:30:153
1537699120,0cyclictest0-21swapper/319:50:143
1537499129,2cyclictest8179-21grep22:45:272
1537499129,2cyclictest6530-21head19:55:262
1537499129,2cyclictest3376-21chrt22:43:102
1537499129,2cyclictest28608-21grep19:35:212
1537499129,2cyclictest27455-21sed21:20:272
1537499129,2cyclictest23417-21grep20:30:252
1537499129,2cyclictest23165-21cut22:55:132
1537499129,2cyclictest22047-21tr22:35:162
1537499129,2cyclictest22047-21tr22:35:162
1537499129,2cyclictest21079-21turbostat00:30:002
1537499129,2cyclictest21058-21sort19:20:172
1537499129,2cyclictest2009-21head20:55:172
1537499129,2cyclictest14906-21expr22:50:142
1537499129,2cyclictest13087-21sed22:10:172
1537499129,2cyclictest11124-21grep21:30:212
1537499129,2cyclictest10384-21sed20:05:172
1537499128,3cyclictest27413-21cron22:00:002
15374991211,1cyclictest0-21swapper/200:00:422
15374991210,2cyclictest32-21ksoftirqd/223:58:222
1536999129,2cyclictest6143-21wget21:05:151
1536999129,2cyclictest5903-21cut22:25:171
1536999129,2cyclictest5544-21tail19:55:171
1536999129,2cyclictest31736-21grep19:40:251
1536999129,2cyclictest28595-21grep19:35:211
1536999129,2cyclictest23531-21wc22:35:281
1536999129,2cyclictest23531-21wc22:35:281
1536999129,2cyclictest18656-21ssh21:15:231
1536999129,2cyclictest13663-21tail20:10:271
1536999129,2cyclictest11916-21tr00:05:141
1536999129,2cyclictest1001-21awk20:55:001
1536999129,2cyclictest1001-21awk20:55:001
1536999128,3cyclictest31045-21cron20:50:001
15369991212,0cyclictest0-21swapper/123:55:211
1536999120,11cyclictest3534-21cut23:40:221
1536599129,2cyclictest9954-21cron23:45:000
1536599129,2cyclictest32301-21cat20:50:190
1536599129,2cyclictest32301-21cat20:50:190
1536599129,2cyclictest25840-21head19:30:170
1536599129,2cyclictest25043-21ssh23:15:160
1536599129,2cyclictest2233-21sort20:55:200
1536599129,2cyclictest21679-21tr00:10:220
1536599129,2cyclictest21536-21grep22:15:170
1536599129,2cyclictest18846-21ssh21:54:510
1536599129,2cyclictest16930-21grep23:10:170
1536599129,2cyclictest1546-21cat19:45:250
1536599129,2cyclictest12611-21cron00:25:000
1536599126,3cyclictest0-21swapper/022:59:200
1537699118,2cyclictest7266-21head21:05:233
1537699118,2cyclictest29394-21expr20:45:143
1537699118,2cyclictest28376-21taskset20:41:163
1537699118,2cyclictest27909-21ssh21:20:443
1537699118,2cyclictest23005-21ssh22:55:093
1537699118,2cyclictest1871-21sort20:55:173
1537699110,5cyclictest8901-21grep20:00:263
1537499118,2cyclictest667-21cut19:45:172
1537499118,2cyclictest5806-21cut22:25:172
1537499118,2cyclictest3965-21ssh00:38:552
1537499118,2cyclictest3651-21ssh21:25:352
1537499118,2cyclictest3650-21cut19:50:222
1537499118,2cyclictest3277-21entropy21:45:172
1537499118,2cyclictest32623-21ssh21:43:312
1537499118,2cyclictest28937-21apache_accesses20:45:112
1537499118,2cyclictest28700-21sort22:00:182
1537499118,2cyclictest24520-21taskset23:15:132
1537499118,2cyclictest15942-21ssh23:10:052
1537499118,2cyclictest12325-21expr20:10:162
1537499117,3cyclictest20384-21cat20:25:202
1537499110,10cyclictest19263-21fgrep23:30:222
1537499110,10cyclictest13653-21sh00:05:282
1537499110,10cyclictest1109-21cat23:20:142
1536999118,2cyclictest7955-21ssh22:45:261
1536999118,2cyclictest27278-21head20:40:161
1536999118,2cyclictest24383-21sed19:25:261
1536999118,2cyclictest23007-21sh00:30:231
1536999118,2cyclictest21618-21date20:30:001
1536999118,2cyclictest19677-21grep19:15:251
1536999118,2cyclictest17196-21ssh21:53:331
1536999118,2cyclictest14448-21ssh22:10:351
1536999118,2cyclictest12891-21ssh00:25:041
1536999118,2cyclictest10870-21cat21:30:191
1536999117,3cyclictest22129-21tr22:15:231
1536999114,3cyclictest0-21swapper/123:45:111
1536999113,6cyclictest18613-21head23:30:171
1536599119,1cyclictest29946-21grep22:00:270
1536599119,1cyclictest12930-21grep22:10:160
1536599119,1cyclictest10728-21cut20:05:200
1536599118,2cyclictest3251-21rm00:00:120
1536599118,2cyclictest32427-21ssh21:24:160
1536599118,2cyclictest32178-21expr23:00:160
1536599118,2cyclictest26895-21grep23:35:190
1536599118,2cyclictest21707-21ssh22:35:130
1536599118,2cyclictest21707-21ssh22:35:130
1536599118,2cyclictest20269-21grep21:35:260
1536599118,2cyclictest19205-21ssh00:28:360
1536599118,2cyclictest19148-21ssh21:15:280
1536599118,2cyclictest18491-21tail20:20:260
1536599118,2cyclictest16542-21cron20:20:000
1536599118,2cyclictest12269-21cut20:10:160
1536599118,2cyclictest1225-21ssh21:25:020
1536599118,2cyclictest11612-21sh00:05:110
1536599118,2cyclictest10534-21ps23:25:220
1536599118,2cyclictest10066-21tr21:30:150
1536599110,7cyclictest0-21swapper/021:55:130
1537699108,1cyclictest3764-21cut21:00:143
1537699108,1cyclictest14622-21grep20:15:153
1537699107,2cyclictest832-21sed19:45:173
1537699107,2cyclictest24198-21cat19:25:253
1537699103,6cyclictest20535-21uname20:25:203
1537499108,1cyclictest92752sleep220:04:262
1537499108,1cyclictest29041-21sh22:20:102
1537499108,1cyclictest27072-21chrt19:33:002
1537499108,1cyclictest17378-21df_abs20:20:152
1537499107,2cyclictest6753-21grep21:05:182
1537499107,2cyclictest14569-21sort20:15:142
1537499106,3cyclictest9410-21grep23:25:142
1537499101,3cyclictest225-21systemd-journal19:41:422
1536999108,1cyclictest10249-21ssh21:10:231
1536999107,2cyclictest8296-21tr20:00:191
1536999107,2cyclictest3698-21wget21:00:141
1536999107,2cyclictest26783-21tail19:30:261
1536999107,2cyclictest20617-21tr19:20:151
1536999106,3cyclictest11163-21wc20:05:261
1536599108,2cyclictest19499-21sed19:15:240
1536599108,1cyclictest4771-21taskset19:55:030
1536599108,1cyclictest3816-21sh22:05:100
1536599108,1cyclictest3343-21cat19:50:190
1536599108,1cyclictest319862sleep019:42:080
1536599108,1cyclictest26508-21cat20:40:000
1536599108,1cyclictest26429-21ssh21:40:040
1536599108,1cyclictest2404-21ssh21:45:040
1536599108,1cyclictest22190-21expr20:30:140
1536599107,2cyclictest27597-21tr23:55:150
1536599107,2cyclictest15820-21tr19:10:140
1536599103,3cyclictest225-21systemd-journal20:44:260
1536599103,3cyclictest20179-21grep20:25:170
153659997,1cyclictest29143-21cat20:45:130
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional