You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-18 - 22:44

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot7s.osadl.org (updated Thu Dec 18, 2025 12:43:47)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2176829177,9sleep00-21swapper/007:05:170
2302228876,8sleep20-21swapper/207:06:432
2307427858,16sleep10-21swapper/107:07:231
2299827764,9sleep30-21swapper/307:06:233
23362993332,1cyclictest0-21swapper/108:00:281
2336999224,17cyclictest26116-21apt-get11:45:143
2336299220,20cyclictest0-21swapper/108:05:121
2335899225,9cyclictest0-21swapper/007:20:140
2336799210,0cyclictest0-21swapper/207:43:402
2336299211,2cyclictest629-21cron07:30:001
2336799200,20cyclictest0-21swapper/208:28:292
2336299200,19cyclictest0-21swapper/110:30:151
2335899205,8cyclictest20853-21munin-run09:20:000
2335899200,18cyclictest0-21swapper/007:25:140
2336799190,1cyclictest0-21swapper/207:31:272
2336799190,18cyclictest0-21swapper/211:05:152
2336999180,15cyclictest0-21swapper/307:20:133
2336299180,12cyclictest0-21swapper/109:50:131
23369991717,0cyclictest0-21swapper/310:30:153
2335899170,14cyclictest0-21swapper/011:45:140
2336299169,6cyclictest31235-21wc10:45:221
2336799150,9cyclictest0-21swapper/211:10:142
2336299158,5cyclictest475-21in:imuxsock07:10:251
2336299150,14cyclictest0-21swapper/111:35:141
2335899150,8cyclictest28124-21sendmail07:20:000
23369991410,3cyclictest11095-21sed11:10:293
23367991410,3cyclictest27102-21idleruntime-cro08:25:002
23367991410,2cyclictest31945-21cat07:25:192
23367991410,2cyclictest26817-21grep11:45:172
23367991410,2cyclictest1122-21wc10:50:232
23367991410,2cyclictest1122-21wc10:50:222
2336799140,14cyclictest0-21swapper/212:36:362
2336299149,3cyclictest15117-21cron07:59:591
23362991410,3cyclictest20150-21ls09:15:261
23362991410,2cyclictest32404-21grep09:40:261
23362991410,2cyclictest28141-21grep10:40:181
23362991410,2cyclictest2279-21cut10:55:121
2336299140,12cyclictest10450-21wc11:10:211
2335899146,7cyclictest4210-21munin-run08:45:010
2335899142,9cyclictest16121-21latency_hist09:10:010
23358991410,2cyclictest13866-21cron11:20:010
49372130,1sleep10-21swapper/108:45:151
2336999139,3cyclictest28141-21grep08:25:213
2336999139,3cyclictest27275-21awk10:40:013
2336999139,3cyclictest1153-21tr07:30:163
2336999133,9cyclictest22700-21apache207:55:243
23369991310,2cyclictest8707-21cut07:45:183
23369991310,2cyclictest6503-21grep12:10:183
23369991310,2cyclictest5362-21sleep09:54:593
23369991310,2cyclictest32529-21sed11:55:253
23369991310,2cyclictest17413-21sh12:35:013
23369991310,2cyclictest12995-21head11:15:223
2336999130,12cyclictest20871-21tr09:20:013
2336999130,12cyclictest1805-21sh08:40:013
2336799134,7cyclictest0-21swapper/209:45:172
23367991310,2cyclictest6505-21tail12:10:182
23367991310,2cyclictest6242-21grep11:00:282
23367991310,2cyclictest4140-21grep12:05:182
23367991310,2cyclictest384-21grep09:44:592
23367991310,2cyclictest23186-21grep08:15:162
23367991310,2cyclictest19880-21cron08:09:592
23367991310,2cyclictest18639-21sh11:30:002
23367991310,2cyclictest16497-21grep09:10:162
23367991310,2cyclictest15692-21grep08:00:132
23367991310,2cyclictest14933-21head11:20:172
23367991310,2cyclictest13741-21grep07:55:182
23367991310,2cyclictest12738-21cron07:54:592
23367991310,2cyclictest11358-21taskset10:05:182
2336299139,3cyclictest2198-21cut08:40:161
2336299139,3cyclictest20058-21cron10:25:001
2336299139,2cyclictest614-21uname08:35:201
2336299139,2cyclictest15600-21grep12:30:131
2336299139,2cyclictest13665-21tail12:25:161
2336299138,4cyclictest7877-21sed08:50:211
23362991310,2cyclictest16810-21tr11:25:161
23362991310,2cyclictest12838-21tr09:00:221
2336299130,12cyclictest21255-21grep08:10:211
2335899139,3cyclictest4527-21tr07:35:220
2335899132,5cyclictest225-21systemd-journal09:05:000
23358991312,1cyclictest0-21swapper/010:13:270
23358991310,2cyclictest6509-21grep12:10:180
23358991310,2cyclictest5034-21grep12:05:260
23358991310,2cyclictest31564-21tail11:55:170
2336999129,2cyclictest7618-21head08:50:183
2336999129,2cyclictest6408-21head09:55:173
2336999129,2cyclictest6397-21grep07:40:173
2336999129,2cyclictest5523-21sh12:10:013
2336999129,2cyclictest28152-21cat09:35:003
2336999129,2cyclictest25695-21grep08:20:153
2336999129,2cyclictest24796-21grep09:25:223
2336999129,2cyclictest22086-21cat09:20:193
2336999129,2cyclictest19834-21sendmail-msp12:40:003
2336999129,2cyclictest19351-21wc08:05:243
2336999129,2cyclictest17508-21cron08:04:593
2336999129,2cyclictest16414-21wc11:25:003
2336999129,2cyclictest1569-21head12:00:173
2336999129,2cyclictest15103-21tr09:05:193
2336999129,2cyclictest14651-21grep10:10:283
2336999129,2cyclictest13301-21grep09:00:273
2336999129,2cyclictest12662-21cron12:25:003
2336799129,2cyclictest8879-21head12:15:172
2336799129,2cyclictest5759-21grep09:55:152
2336799129,2cyclictest30497-21sed08:30:162
2336799129,2cyclictest28167-21cut10:40:172
2336799129,2cyclictest27283-21tr10:40:012
2336799129,2cyclictest25858-21awk11:45:002
2336799129,2cyclictest2345-21grep08:40:152
2336799129,2cyclictest21852-21grep09:20:172
2336799129,2cyclictest21389-21head08:10:232
2336799129,2cyclictest19220-21df09:15:172
2336799129,2cyclictest14663-21grep10:10:282
2336799129,2cyclictest13025-21wc11:15:232
2336799128,3cyclictest906-21grep08:35:242
2336299129,2cyclictest9825-21grep12:15:271
2336299129,2cyclictest6521-21grep12:10:181
2336299129,2cyclictest5740-21grep11:00:221
2336299129,2cyclictest5122-21grep12:05:271
2336299129,2cyclictest30468-21cron07:25:001
2336299129,2cyclictest28481-21uname08:25:241
2336299129,2cyclictest28076-21cron07:20:001
2336299129,2cyclictest27561-21grep09:30:271
2336299129,2cyclictest26353-21tr11:45:151
2336299129,2cyclictest25239-21tr08:20:121
2336299129,2cyclictest24689-21tr09:25:201
2336299129,2cyclictest23686-21tail08:15:191
2336299129,2cyclictest20601-21tr10:25:161
2336299129,2cyclictest1647-21sed09:45:181
2336299129,2cyclictest16463-21grep09:10:151
2336299129,2cyclictest10992-21cut10:05:151
2336299128,3cyclictest31482-21sort11:55:171
2336299127,3cyclictest29846-21cut08:30:131
2336299120,3cyclictest0-21swapper/109:39:591
2336299120,1cyclictest0-21swapper/108:55:121
2335899129,2cyclictest8831-21tail07:45:200
2335899129,2cyclictest6174-21df07:40:160
2335899129,2cyclictest6126-21grep08:45:250
2335899129,2cyclictest6111-21df09:55:150
2335899129,2cyclictest2981-21sh09:50:000
2335899129,2cyclictest275422sleep010:40:140
2335899129,2cyclictest19294-21sort08:05:240
2335899129,2cyclictest15256-21cat11:20:200
2335899129,2cyclictest13009-21tr12:25:130
2335899128,3cyclictest14622-21sed07:55:260
2335899128,2cyclictest8239-21grep12:15:140
2335899120,11cyclictest2574-21tail07:30:270
2336999119,1cyclictest10650-21grep10:05:133
2336999118,2cyclictest9970-21head08:55:163
2336999118,2cyclictest8967-21sshd11:08:103
2336999118,2cyclictest4867-21tr11:00:153
2336999118,2cyclictest4378-21date08:45:003
2336999118,2cyclictest31517-21cat09:40:183
2336999118,2cyclictest30722-21latency08:30:183
2336999118,2cyclictest29902-21wc11:50:273
2336999118,2cyclictest24620-21wc07:10:153
2336999118,2cyclictest24024-21tr11:40:163
2336999118,2cyclictest22854-21sed11:35:263
2336999118,2cyclictest20494-21cat08:10:163
2336999118,2cyclictest18455-21cron09:15:003
2336999118,2cyclictest16532-21apache_processe11:25:143
2336999118,2cyclictest15001-21seq12:29:353
2336999118,2cyclictest12186-21wc07:50:263
2336799118,2cyclictest9966-21grep08:55:162
2336799118,2cyclictest8649-21taskset08:51:202
2336799118,2cyclictest6154-21grep08:45:252
2336799118,2cyclictest5201-21seq07:37:382
2336799118,2cyclictest30561-21grep10:45:162
2336799118,2cyclictest29694-21wc07:20:222
2336799118,2cyclictest28529-21aten2_r7power_e11:50:152
2336799118,2cyclictest28451-21grep09:35:142
2336799118,2cyclictest27681-21grep07:15:252
2336799118,2cyclictest25252-21grep07:10:212
2336799118,2cyclictest2456-21grep12:00:272
2336799118,2cyclictest23452-21cut10:30:182
2336799118,2cyclictest21876-21grep11:35:172
2336799118,2cyclictest20594-21grep10:25:152
2336799118,2cyclictest19129-21tr11:30:152
2336799118,2cyclictest15994-21taskset09:08:592
2336799117,3cyclictest20080-21awk10:25:002
2336799117,3cyclictest16094-21grep12:30:162
2336799111,2cyclictest11604-21apache_accesses09:00:122
2336799110,1cyclictest0-21swapper/210:55:202
2336299118,2cyclictest7635-21cut11:05:181
2336299118,2cyclictest3428-21grep07:35:151
2336299118,2cyclictest25874-21tail10:35:171
2336299118,2cyclictest17215-21tail10:15:251
2336299118,2cyclictest15763-21chrt09:05:591
2336299118,2cyclictest13210-21tr10:10:161
2336299118,2cyclictest1226-21which10:50:241
2336299118,2cyclictest1226-21which10:50:231
2336299118,2cyclictest1199-21tr07:30:161
2336299118,2cyclictest11427-21grep07:50:171
2336299118,2cyclictest10140-21sleep10:05:001
2336299117,3cyclictest17813-21basename12:35:131
2335899118,2cyclictest9625-21grep10:00:260
2335899118,2cyclictest8145-21gzip08:50:220
2335899118,2cyclictest4023-21head09:50:160
2335899118,2cyclictest354-21apt-get09:44:590
2335899118,2cyclictest32766-21grep08:35:180
2335899118,2cyclictest31355-21grep08:30:250
2335899118,2cyclictest28076-21tr08:25:220
2335899118,2cyclictest26147-21cat09:30:150
2335899118,2cyclictest25100-21date10:35:000
2335899118,2cyclictest24797-21taskset10:34:040
2335899118,2cyclictest23706-21uname08:15:190
2335899118,2cyclictest21064-21head10:25:180
2335899118,2cyclictest19433-21chrt12:35:330
2335899118,2cyclictest17656-21tr11:25:210
2335899118,2cyclictest17578-21grep09:10:230
2335899118,2cyclictest17524-21cat12:35:000
2335899118,2cyclictest1475-21grep10:50:260
2335899118,2cyclictest1475-21grep10:50:260
2335899118,2cyclictest10938-21df10:05:140
2335899118,2cyclictest10516-21gpgv07:50:000
2335899117,3cyclictest24287-21head09:25:160
2335899117,3cyclictest22740-21apache212:00:230
2335899111,8cyclictest30435-21munin-run09:39:590
2336999108,1cyclictest4597-21hostname08:45:133
2336999108,1cyclictest29969-21seq10:45:133
2336999108,1cyclictest28192sleep309:48:033
2336999108,1cyclictest24669-21taskset08:19:413
2336999108,1cyclictest18416-21cat10:20:143
2336999107,2cyclictest8988-21basename12:15:183
2336999107,2cyclictest32618-21expr10:50:163
2336999107,2cyclictest32618-21expr10:50:153
2336999107,2cyclictest31418-21expr07:25:153
2336999107,2cyclictest28686-21expr09:35:153
2336999107,2cyclictest27547-21apt-get10:40:143
2336999107,2cyclictest20983-21grep10:25:173
2336999107,2cyclictest15753-21egrep10:15:133
2336999107,2cyclictest10117-21cron10:04:593
2336799108,1cyclictest9932-21taskset10:02:392
2336799108,1cyclictest9610-21sed07:45:292
2336799108,1cyclictest32635-21taskset11:55:562
2336799108,1cyclictest26086-21grep09:30:152
2336799108,1cyclictest24358-21which09:25:172
2336799108,1cyclictest17406-21chrt10:16:432
2336799108,1cyclictest11252-21grep12:20:202
2336799107,2cyclictest4326-21cat09:50:192
2336799107,2cyclictest13007-21grep12:25:132
2336299108,1cyclictest72492sleep109:55:261
2336299108,1cyclictest5769-21cut07:40:151
2336299108,1cyclictest2913-21taskset12:04:491
2336299108,1cyclictest12349-21chrt12:21:001
2336299108,1cyclictest11730-21seq11:15:101
2336299107,2cyclictest30385-21chrt11:52:301
2336299107,2cyclictest21369-21tr09:20:151
2336299107,2cyclictest19148-21tr11:30:151
2336299106,3cyclictest15989-21taskset11:21:341
2335899108,1cyclictest5102-21expr11:00:170
2335899108,1cyclictest268652sleep008:22:050
2335899108,1cyclictest197822sleep010:21:250
2335899108,1cyclictest17550-21ls08:05:010
2335899107,2cyclictest9857-21fschecks_time08:55:150
2335899107,2cyclictest3895-21grep10:55:250
2335899107,2cyclictest25865-21cat07:15:000
2335899107,2cyclictest25358-21grep11:40:300
2335899107,2cyclictest19657-21head11:30:180
2335899107,2cyclictest12088-21cat12:20:290
2335899100,9cyclictest9355-21taskset11:10:100
2335899100,9cyclictest7137-21cut11:05:160
233699999,0cyclictest0-21swapper/307:35:133
233699998,1cyclictest19037-21hostname11:30:153
233699997,1cyclictest25975-21apache_volume07:15:123
233699997,1cyclictest2429-21egrep10:55:133
233629997,1cyclictest8749-21grep07:45:201
233629996,2cyclictest24643-21cat11:40:221
233589997,1cyclictest31011-21sed10:45:190
233589997,1cyclictest29579-21grep11:50:210
233589997,1cyclictest22651-21tr11:35:230
233589997,1cyclictest20292-21cut08:10:150
233589997,1cyclictest16343-21tail10:15:150
233589996,2cyclictest21724-21fschecks_count09:20:180
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional